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When  Government  drawings,  specifications,  or  other  data  are  used  for  any 
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the  holder,  or  any  other  person  or  corporation;  or  as  conveying  any  rights  or 
permission  to  manufacture,  use,  or  sell  any  patented  invention  that  may  in  any 
way  be  related  thereto. 

The  Public  Affairs  Office  has  reviewed  this  paper,  and  it  is  releasable  to  the 
National  Technical  Information  Service,  where  it  will  be  available  to  the 
general  public,  including  foreign  nationals. 

This  paper  has  been  reviewed  and  is  approved  for  publication. 
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Training  Systems  Division 


RODGER  D.  BALLENTINE,  Colonel,  USAF 
Chief,  Training  Systems  Division 


This  technical  paper  is  printed  as  received  and  has  not  been  edited  by  the 
AFHRl  Technical  Editing  staff.  The  opinions  expressed  herein  represent  those 
of  the  author  and  not  necessarily  those  of  the  United  States  Air  Force. 


REPORT  DOCUMENTATION  PAGE 
_ 

form  Approved  1 

om  No.  0704-01  tt  I 

Putdc raporttw  bwdtn  for  dm coMcoon  or  information  t  aiomatad  10  rnnnqt  i  i— ' - •‘-j  '-rrr~'nn.  mrctilni  nwni  ' 

mOwum and  «WU»»S  ox data naadad.  and  comofotmg  UK ramaanng  tf»a  coddcoon  y  fofonwadoii.  fond  command  raoardma  tfm  bwdan  aadmata  arm-  'mS •maaatiiZ 
coNkdm  q*  information.  mdgdiifo  MMattiona  for  rt duong  ttw>  burdan.  la  WaWMigiaa  iHadbbartari  lawn.  Hwaww  for  Inform  How  Odarattod  tad  Sam.  llttMNfoS 
DmHlghMr,  SuItt  1294.  ArUnglon.  VA  22202-4302,  and  to  Tha  Offlca  Ot  Managamani  and  dudgat.Padanaort(  daductton  Prolact 107044  tdmiiailMMngaoil.  QCOOwb 

4.  TITLE  A NO  SUBTITLE 

Advanced  On-the-job  Training  System: 

System  Specification 

5.  FUNDING  NUMBERS 

C  -  F3361 5-84-C-0059 

PR  -  2557 

TA  -  00 

HU  -  02 

«.  AUTHORS)  - 

7.  PERFORMING  ORGANIZATION  NAME(S)  ANO  ADORE  s's/tS) 

Douglas  Aircraft  Company 

A  Division  of  McDonnell  Douglas  Corporation 

2450  South  Peoria 

Aurora,  Colorado  60014 

sT  PERFORMING  ORGANIZATION 

REPORT  NUMBER 

9.  SPONSORING /MONITORING  AGENCY  NAME(S)  ANO  AOORESS<ES) 

Training  Systems  Division 

Air  Force  Human  Resources  Laboratory 

Brooks  Air  Force  Base,  Texas  78235-5601 

10.  SPONSORING  /  MONITOIUNG 

AGENCY  REPORT  NUMBER 

AFWL-TP-89-93 

11.  SUPPLEMENTARY  NOTES 

12a.  DISTRIBUTION  /  AVAILABILITY  STATEMENT 

Approved  for  public  release:  distribution  Is  unlimited. 

12b.  DISTRIBUTION  COOE 

13.  ABSTRACT  (Maximum 200  worth) 

V  c~ 

The  System  Specification  for  the  Advanced  On^the^job  Training  System  (A0TS)  establishes  the 
performance,  design,  development  and  test  requirements  for  the  prototype  AOTS.  The  purpose  of  the  AOTS 
was  to  test  a  design  concept  for  Improving  current  practice  In  the  Air  Force  On-the-Job  Training  (OJT) 
program.  The  AOTS  Is  a  computer-based,  training  management,  training  development  and  delivery,  and 
training  evaluation  system  especially  designed  to  enhance  OJT  activities  In  the  field.  kj£4/j*o\dz' 

0 

14.  SUBJECT  TERM  U. 

^advanced  or/the-job  training  system^ 

system  specification ,  ( _  - 

”  i 

IB.  SfCUMTV  O&SSMCATKM 
OP  tms  PAGE 

Unclassified 

1*.  security  dAiSmbtfiON 

OP  ABSTRACT 

Unclassified 

AFWU.  Technical  Paper  89-93 


May  1990 


ADVANCED  ON-THE-JOB  TRAINING  SYSTEM: 
SYSTEM  SPECIFICATION 


Douglas  Aircraft  Company 
A  Division  of  McDonnell  Douglas  Corporation 
2450  South  Peoria 
Aurora,  Colorado  80014 


TRAINING  SYSTEMS  DIVISION 
Brooks  Air  Force  Base,  Texas  78235-5601 


Reviewed  and  submitted  for  publication  by 

Jack  L.  Blackhurst,  Major,  USAF 
Chief,  Advanced  On-the-job  Training  System  Program 


This  publication  is  primarily  a  working  paper.  It  is  published  solely  to  document  work  performed 


SUMMARY 


The  Advanced  On-the-job  Training  System  (AOTS)  was  an  Air  Staff  directed, 
AFHRL  developed,  prototype  system  which  designed,  developed,  and  tested  a 
proof-of-concept  prototype  AOTS  within  the  operational  environment  of  selected 
work  centers  at  Bergstrom  AFB,  Texas,  and  Ellington  ANGB,  Texas,  from  August 
1985  through  31  July  1989.  This  System  Specification  establishes  the  perfor¬ 
mance  design,  development  and  test  requirements  for  the  prototype  AOTS.  The 
purpose  of  the  AOTS  was  to  test  a  design  concept  for  improving  current  prac¬ 
tices  in  the  Air  Force  On-the-Job  Training  (OJT)  program.  The  AOTS  is  a 
computer-based,  training  management,  training  development  and  delivery,  and 
training  evaluation  system  especially  designed  to  enhance  OJT  activities  in 
the  field. 
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PREFACE 


This  paper  was  prepared  by  Oouglas  Aircraft  Company,  the  AOTS  development 
contractor,  under  Government  Contract  Number  F33615-84-C-0059.  The  AFHRL 
Work  Unit  number  for  the  project  Is  2557-00-02.  The  primary  office  of  res¬ 
ponsibility  for  management  of  the  work  unit  is  the  Air  Force  Human  Resources 
Laboratory,  Training  Systems  Division,  and  the  Air  Force  AOTS  manager  is 
Major  Jack  Blackhurst. 
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1  SCOPE 

This  specification  establishes  the  performance  design,  development  and  test  requirements  for  the 
prototype  Advanced  On-the-job  Training  System  (AOTS).  The  purpose  of  the  AOTS  is  to  test  a 
design  concept  for  improving  current  practice  in  the  Air  Force  On-the-Job  Training  (OJT) 
program.  The  AOTS  is  a  computer-based,  training  management,  training  development  and 
delivery,  and  training  evaluation  system  especially  designed  to  enhance  OJT  activities  in  the 
field.  The  prototype  will  be  installed  and  tested  at  Bergstrom  AFB,  Texas  and  Ellington  ANGB, 
Texas. 

The  AOTS  program  shall  consist  of  three  phases.  They  are  referred  to  as  follows: 

Phase  I  Preliminary  Design  Phase 

Phase  II  Detail  Design  and  Development  Phase 

Phase  III  System  Level  Test  and  Evaluation  (SLT&E) 

Each  of  these  three  phases  also  correspond  to  calendar  time.  Phase  I  before  Phase  II  and  Phase 
III  after  Phase  II.  A  Phase  IV  designation  was  applied  to  the  Training  Development  and  Delivery 
Subsystem  (i'DDS)  as  it  was  not  part  of  the  initial  contractor  effort.  When  this  element  was 
added  to  the  contractor’s  effort,  it  was  incorporated  into  Phase  II  and  III  calendar  time  too. 
After  its  incorporation,  the  designation  Phase  IV  no  longer  was  applicable. 


2  APPLICABLE  DOCUMENTS 


The  following  documents,  of  the  exact  issue  shown,  form  a  part  of  this  specification  to  the  extent 
specified  herein.  In  the  event  of  conflict  between  the  documents  referenced  herein  and  the 
detailed  contents  of  this  specification,  the  contents  of  this  specification  shall  be  considered  as 
the  superseding  requirement. 


2.1  Government  Documents. 

The  following  Government  documents  are  applicable  to  this  specification. 


2. 1 . 1  Specifications. 


70S647100 
31  Oct  1989 

70S6472 00 
1  Apr  1986 

70S647201 
1  Apr  1986 

70S647202 
1  Apr  1986 


Prime  Item  Development  Specification  for  the  Man¬ 
agement  Subsystem 

Prime  Item  Development  Specification  for  the  Training 
Development  and  Delivery  Subsystem 

The  Training  Development  Component  Critical  Item 
Specification 

The  Training  Delivery  Component  Critical  Item  Speci¬ 
fication 


70S647300  Prime  Item  Development  Specification  for  the  EvalU' 

31  Oct  1989  ation  Subsystem 


70S647400 
17  Apr  1986 


Prime  Item  Development  Specification  for  the  Computer 
Support  Subsystem 
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70S647401 

1  Jun  1988 

Critical  Item  Development  Specification  for  the  Hard¬ 
ware  Component  of  the  Computer  Support  Subsystem 

70S64741 i 

Part  1 

17  Apr  1986 

The  Management  Subsystem  Computer  Program  Con¬ 
figuration  Item  Development  Specification 

70S46741 1 

Part  2 

23  Jun  1989 

The  Management  Subsystem  Computer  Program  Con¬ 
figuration  Item  Product  Specification 

70S647412 

Part  1 

17  Apr  1986 

The  Training  Development  Delivery  Subsystem  Com¬ 
puter  Program  Configuration  Item  Development  Speci¬ 
fication 

70S647413 

Part  1 

17  Apr  1«86 

The  Evaluation  Subsystem  Computer  Program  Config¬ 
uration  Item  Development  Specification 

70S647413 

Part  2 

23  Jun  1989 

The  Evaluation  Subsystem  Computer  Program  Config¬ 
uration  Item  Product  Specification 

70S647414 

Part  1 

17  Apr  1986 

The  System  Support  Computer  Program  Configuration 
Item  Development  Specification 

70S647414 

Part  2 

23  Jun  1989 

The  System  Support  Computer  Program  Configuration 
Item  Product  Specification 

70S647500 

28  Mar  1986 

Prime  Item  Development  Specification  for  the  Personnel 
and  Support  Subsystem 

Standards. 

MIL-STD-421C 

12  Jun  1981 

Definitions  of  Effectiveness  Terms  for 

Reliability,  Maintainability,  Human  Factors  and 

Safety 

MIL-STD-470A 

3  Jan  1983 

Maintainability  Program  Requirements  for 

Systems  and  Equipment 

MIL-STD-471A 

8  Dec  1978 

Maintainability  V erif ication/Demonstration 
/Evaluation 

MIL-STD-483 

Notice  2 

21  Mar  1979 

Configuration  Management  for  Systems, 

Equipment,  Munitions,  and  Computer  Programs 

MIL-STD-785B 

15  Sept  1980 

Reliability  Program  for  Systems  and  Equipment 
Development  Production 

P*0*2 
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MIL-STD-1472C  Human  Engineering  Design  Criteria  for 

i  Sept.  1983  Military  Systems,  Equipment  and  Facilities 


MIL-STD-1815A  Ada  Programming  Language 

22  Jan  1983 


2.1.3  Other  Publications. 


MIL-H-46855A  Human  Engineering  Requirements  for  Military 

2  May  1972  Systems,  Equipment,  and  Facilities 


AOTS  Software  Development 
Plan 

7  Apr  1986 

Computer  Program  Test 
Report 
19  Dec  1988 

Computer  Programming  Stan¬ 
dards  Manual 
15  Sep  1986 

Configuration  Management 
Plan 

18  Jul  1988 

Final  Technical  Report 
31  Oct  1989 


Human  Engineering  Plan 
27  Jan  1987 


Interim  Technical  Report 
6  Jan  1987 


Maintainability  Program  Plan 
I  Oct  1986 


Master  Test  Plan 
31  Aug  1987 

Reliability  Program  Plan 
1  Oct  1986 


Site  Preparation  and  Installa¬ 
tion  Plan 
25  Aug  1986 

System/Design  Trade  Study 
Report(s) 

14  Apr  1986 

System  Engineering  Manage¬ 
ment  Plan 
30  Sep  1986 


Prepared  by  Douglas  Aircraft  Company,  a  division  of 
McDonnell  Douglas  Corporation,  under  contract 

F336 1 5-84-C-0059 

Prepared  by  Douglas  Aircraft  Company,  a  division  of 
McDonnell  Douglas  Corporation,  under  contract 

F336 1 5-84-C-0059 

Prepared  by  Douglas  Aircraft  Company,  a  division  of 
McDonnell  Douglas  Corporation,  under  contract 

F33615-84-C-0059 

Prepared  by  Douglas  Aircraft  Company,  a  division  of 
McDonnell  Douglas  Corporation,  under  contract 

F3361 5-84-C-0059 

Prepared  by  Douglas  Aircraft  Company,  a  division  of 
McDonnell  Douglas  Corporation,  under  contract 

F336 1 5-84-C-0059 

Prepared  by  Douglas  Aircraft  Company,  a  division  of 
McDonnell  Douglas  Corporation,  under  contract 

F3 36 1 5-84-C-0059 

Prepared  by  Douglas  Aircraft  Company,  a  division  of 
McDonnell  Douglas  Corporation,  under  contract 

F336 1 5-84-C-0059 

Prepared  by  Douglas  Aircraft  Company,  a  division  of 
McDonnell  Douglas  Corporation,  under  contract 

F336 1 5-84-C-0059 

Prepared  by  Douglas  Aircraft  Company,  a  division  of 
McDonnell  Douglas  Corporation,  under  contract 

F336 1 5-84-C-0059 

Prepared  by  Douglas  Aircraft  Company,  a  division  of 
McDonnell  Douglas  Corporation,  under  contract 

F336 1 5-84-C-0059 

Prepared  by  Douglas  Aircraft  Company,  a  division  of 
McDonnell  Douglas  Corporation,  under  contract 

F3361 5-84-C-0059 

Prepared  by  Douglas  Aircraft  Company,  a  division  of 
McDonnell  Douglas  Corporation,  under  contract 

F33615-84-C-0059 

Prepared  by  Douglas  Aircraft  Company,  a  division  of 
McDonnell  Douglas  Corporation,  under  contract 

F336 1 5-84-C-0059 
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AFR  12-35 
June  1985 

AFR  30-17 
15  Jun  1977 

AFR  50-1 
Aug  1987 

AFR  50-8 
Aug  1984 

AFM  30-3 
Vol  4. 

Mar  1986 

AFM  30-130 
Vol.  1 
Oct  1986 

AFM  50-2 
Jul  1986 

AFM  66-279 
Vol.  17 
Jan.  1987 

AFR  205-1 
7  Dec  1982 


Air  Force  Privacy  Act  Program 

Safeguarding  Controlled  Item  (Test  Materials) 

Information 

Ancillary  Training  Program 

Policy  and  Guidance  for  Instructional  System  Development 
(ISD) 

Personnel  Accounting  Symbol  System 


Base  Level  Military  Personnel  System 


Instructional  System  Development 

Core  Automated  Maintenance  System  (CAMS  ISD: 
G054/FS(PA)  Training  Management 

Information  Security  Program 


2.2  Non-Government  Documents. 

None  are  required  for  this  specification. 


3  REQUIREMENTS 
3.1  System  definition. 

The  Advanced  On-the-job  Training  System  (AOTS)  is  a  prototype,  proof -of -concept,  computer- 
based  training  system  that  shall,  when  developed,  apply  automated  support  to  increase  the 
efficiency  ana  effectiveness  of  the  current  Air  Force  On-The-Job  Training  (OJT)  system.  The 
AOTS  shall  provide  improved  capabilities  to: 


a.  Define  the  performance  and  trainingrequirements  for  entire  Air  Force  Specialties 
(AFSs),  groups  of  members  within  ArSs,  groups  of  members  across  AFSs,  and  indi¬ 
vidual  trainees; 

b.  Manage  training  progress  through  the  attainment  of  task  and  duty  position  quali¬ 
fication  for  individuals  throughout  the  entire  periods  of  their  Air  Force  careers; 

c.  Provide  list(s)  of  resources  required  to  develop  and  deliver  training,  and  to  evaluate 
trainee  knowledge  and  performance  skills; 

d.  Develop  instructional  materials  and  evaluation  instrumentation; 

e.  Deliver  instruction  and  trainee  evaluation;  and 

f.  Evaluate  the  effectiveness  of  the  training  system. 


The  AOTS  shall  rely  on  interfaces  with  existing  and  future  Air  Force  personnel,  training,  and 
resource  management  systems  to  further  facilitate  efficient  and  effective  operation.  Existing 
Air  Force  Systems  with  which  the  AOTS  shall  interface  include: 


p*qm 
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a.  The  Personnel  Data  System  (PDS); 

b.  The  Core  Automated  Maintenance  System  (CAMS); 

This  specification  applies  to  a  prototype  proof-of-concept  AOTS  to  be  developed  and  tested  at 
Bergstrom  AFB,  and  Ellington  ANGB,  both  located  in  Texas.  The  AOTS  prototype  shall  be 
developed  and  tested  within  Air  Force  active  duty,  reserve,  and  Air  National  Guard  forces.  The 
participating  Air  Force  Specialties  for  the  prototype  AOTS  are  Security  Police,  Aircraft  Main¬ 
tenance,  Aircraft  Jet  Engine  Maintenance  and  Personnel.  Workcenter  supervisors,  trainers, 
trainees,  training  managers  and  commanders  shall  have  access  to  the  prototype  AOTS  through 
terminals  located  within  their  respective  work  areas. 

The  AOTS  shall  include  five  subsystems.  These  subsystems  are: 

a.  The  Management  Subsystem  (Specification  70S647100); 

b.  The  Training  Development  and  Delivery  Subsystem  (Specification  70S647200); 

c.  The  Evaluation  Subsystem  (Specification  70S647300); 

d.  The  Computer  Support  Subsystem  (Specification  70S647400);  and 

e.  The  Personnel  and  Logistics  Support  Subsystem  (Specification  70S647S00). 

The  Computer  Program  Configuration  Items  (CPCIs)  to  support  the  AOTS  subsystems  are: 

a.  The  Management  CPCI  (Specification  70S647411); 

b.  The  Training  Development  and  Delivery  CPCI  (Specification  70S647412); 

c.  The  Evaluation  CPCI  (Specification  70S647413);  and 

d.  The  System  Support  CPCI  (Specification  70S647414). 


3.1.1  General  Information. 

AOTS  shall  prototype  functional  requirements  to  manage  and  administer  OJT  for  an  entire  major 
Air  Force  installation  (the  prototype  test  is  constrained  to  a  sampling  of  AFSs  and  two  bases  - 
Bergstrom  AFB  and  Ellington  ANGB).  It  shall  have  the  capacity  to  accommodate  training  loads 
of  up  to  eight  hundred  trainees.  The  AFSs  and  workcenters  that  have  been  selected  to  participate 
in  the  development  and  test  of  the  prototype  AOTS  are  representative  of  those  found  at  any 
major  Air  Force  installation. 


a.  The  prototype  system  shall  consist  of  a  central  computer  with  dedicated  lines  extending 
to  the  participating  workcenters  to  facilitate  system  access  via  terminals.  Terminals 
shall  also  be  provided  for  members  of  an  Air  Force  instructional  systems  team  and 
contractor  personnel  to  facilitate  development  and  system  test  efforts. 

b.  The  prototype  system  shall  be  capable  of  performing  the  same  types  of  training 
development,  delivery,  management  and  evaluation  functions  required  of  a  larger 
operational  system,  except  that  electronic  interfaces  with  external  computer  systems 
shall  be  simulated.  The  general  capabilities  that  shall  be  provided  by  the  prototype 
system  are  listed  below,  beneath  the  applicable  functional  area  categories. 

1.  Training  Management 

(a)  Automated  identification  of  tasks  required  for  each  AFS  and  duty 
position,  and  for  specified  groups  of  individuals 

(b)  Automated  identification  of  training  needed  by  each  individual  to  qualif  y 
in  his/her  position 
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(c)  Automated  records  of  training  status  and  completions  spanning  indi¬ 
viduals'  entire  Air  Force  careers 

(d)  Automated  prioritizing  of  development,  training  and  evaluation 
requirements 

(e)  Automated  identification  of  resources  required  for  development  of 
training  and  evaluation  materials,  and  for  delivery  of  training  and 
evaluations 

(f)  Automated  assignment  of  training  requirements,  and  scheduling  training 
and  evaluation  events 

(g)  Automated  generation  of  reports  relevant  to  trainee  progress  status 

2.  Training  Development 

(a)  Automated  task  analysis 

(b)  Automated  behavioral  objective  authoring 

(c)  Automated  test  item  authoring 

(d)  Automated  Computer  Assisted  Instruction  (CAI)  lesson  authoring 

3.  Training  Delivery 

(a)  On-line  delivery  of  lessons 

(b)  On-line  delivery  of  graphics  and  simulations  and  Interactive  Video  Disk 
(IVD) 

(c)  On-line  and  off-line  delivery  of  trainee  evaluations 

(d)  Interface  with  training  delivery  agencies  external  to  the  AOTS  for 
scheduling  of  events  and  collecting  trainee  progress  data 

4.  Training  System  Evaluation 

(a)  Automated  selection  of  personnel  and  tasks  for  Quality  Control  evalu¬ 
ations  to  determine  the  ef  f ectiveness  of  training  received  and  to  determine 
individual  retention  of  proficiency 

(b)  Automated  generation  of  reports  relevant  to  AOTS  efficiency  and 
effectiveness,  and  the  effectiveness  of  individual  OJT  programs 

(c)  Automated  analysis  of  knowledge  test  items  and  performance  evaluation 
instruments  to  determine  their  validity  and  the  validity  of  behavioral 
objectives  and  training  materials. 


3.1.2  Mission. 


The  prototype  AOTS  shall  prove  the  feasibility  of  enhancing  OJT  with  computer  support  by: 

a.  Focusing  attention  on  job  task  proficiency  through  promoting  complete  specification 
of  the  tasks  required  to  be  performed  in  a  given  duty  position,  and  by  promoting 
adequate  definition  of  the  training  required  to  become  fully  position  qualified; 


b.  Providing  workable  and  cost-effective  methods  for  evaluating  task  performance  in 
an  operational  setting; 
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c.  Easing  the  inherent  difficulties  of  using  o|>erational  equipment  for  OJT  by  more 
efficient  scheduling  of  training  and  evaluation; 

d.  Addressing  the  problem  of  limited  availability  of  technically  qualified  personnel  to 
act  as  trainers,  and  the  limited  training  skills  of  such  personnel  when  they  are 
available  by  providing  automated  support  for  training  activities; 

e.  Easing  the  paperwork  burden  associated  with  scheduling  OJT  opportunity,  tracking 
training  progress,  evaluating  task  and  associated  test  performance,  and  managing  the 
flow  of  training  data; 

f.  Addressing  the  difficulty  of  adapting  modern  curriculum  development  and  delivery 
procedures  to  the  job  site  environment  (via  a  through  e  above). 


3.1.3  Threat. 


This  paragraph  is  not  applicable  to  this  specification. 


3.1.4  System  diagrams. 

This  paragraph  incorporates  the  system-level  diagrams.  Figures  1  and  2  provide  a  system  level 
view  of  the  AOTS  as  a  training  system  and  as  a  computer  system.  Figures  3  thru  9  comprise  a 
visual  table  of  contents  for  AOTS  as  well  as  a  functional  hierarchy  description.  Figure  10  is 
the  specification  tree  for  AOTS,  depicting  all  the  specifications  that  shall  be  generated  during 
the  development  and  implementation  of  the  system. 
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Figure  i.  ACTS  Training  System  Diagram 
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NOTE:  *  Interactive  Video  Disk  (IVD)  I*  optional  and  not  on  all  terminals  In  the  prototype  AOTS  workcenters. 
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1  MANAGEMENT  subsystem 


Figure  3.  Management  Subsystem 
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I  2  EVALUATION  SUOSYSTEM 


■  2.1  EVALUATION  INSTRUMENT  MANAGEMENT  COMPONENT  j 

— |  2.1.1  DEVELOP  BEHAVIORAL  OBJECTIVES  ~1 

—  2.1.2  PLAN  EVALUATION  MATERIALS 

— 1  2.1.3  MAINTAIN  EVALUATION  MATERIALS  j 

-T  2.1.3.1  MAINTAIN  TEST  ITEM  BANK 
H  2- 1-3.2  MAINTAIN  TESTS 
— {  2~2  PERFORMANCE  EVALUATION  COMPONENT 

2^1  ADMINISTER  KNOWLEDGE  TEST  OFFUNE 


-4  2JZ2.  ADMINISTER  EVALUATIONS  OFFLINE 
H  2.2.3  SCORE  TEST  AND  PROVIDE  FEEDBACK 

~\  2^.4  ACCOUNT  FOR  OFFUNE  EVALUATION  MATERIALS  | 

2J3  TRAINING  QUALITY  CONTROL  COMPONENT  ~  | 

H  23.1  ACCOMPUSH  QUALITY  CONTROL  EVALUATION  EUGIBIUTY  SELECTION 

H  2-3-2  INITIATE  QUALTtY  CONTROL  EVALUATION  | 

H  2-4  SYSTEM  EVALUATION  COMPONENT  ) 

H  2-<1  GENERATE  STANDARD  REPORTS  | 

M  2.4.2  GENERATE  AD  HOC  REPORTS  | 


Figure  4.  Evaluation  Subsystem 
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Figure  5.  Computer  Support  Subsystem 
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Figure  «.  Personnel  end  Logistics  Support  Subsystem 


I  OS  | 


SPECIFICATION  NUMBER  70S647000 
COOE  (DENT  NO.  76301 
DATE  31  October  ISSS 


TRAINING  DCVliLOl'MUNI  &  DGLIVCHY  SUBSYSTEM 


Figure  7.  Training  Development  and  Oefivery  Subsystem 
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— 1  5  1  2. 1 . 1.5  PRINT  UTILITIES  j 

— j  5. 1.2. 1.1. 5.1  SIMULATION  PRINT 

H~S.1.2.1.1.S.2CAI  LESSON  PRINT 

\  5.12.1.2  ESTABLISH  BRANCHING  LOGIC 

H  5.1.2.121  PRE-FRAME  LOGIC 

H  5.12122  AFTERFRAME  LOGIC 

H  5.12.122  RESPONSE  LOGIC 

S.  1 .3  INSTRUCTIONAL  MATERIALS  VALIDATION 


5.1.4  INSTRUCTIONAL  DATA  COLLECTION 


5.2  TRAINING  DELIVERY  COMPONENT 


5.2.1  ACCESS  CONTROL 


H  5.22  MATERIALS  STORAGE  DISTRIBUTION 
H  5.2.3  TRAINING  DEUVERY  DATA  COLLECTION 


Figure  9.  Training  Development  and  Delivery  Subsystem  -  continued 
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Figure  10.  AOTS  Specification  Tree 
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3.1.5  Interlace  definitions . 

This  section  defines  the  interfaces  that  shall  be  required  for  the  AOTS  to  function  as  a 
computer-based  training  system  within  the  Air  Force  operational  environment.  Definitions  are 
included  for  both  the  interfaces  that  shall  occur  between  the  AOTS  and  Air  Force  systems  and 
organizations  external  to  the  AOTS,  and  the  interface  between  the  AOTS  subsystems. 

3. 1 .5. 1  Interfaces  between  the  AOTS  and  external  entities. 

The  external  entities  with  which  the  AOTS  shall  interface  are  depicted  in  Figure  1 1.  Interfaces 
that  shall  be  established  between  the  AOTS  and  other  Air  Force  computer-based  systems  and 
organizations  are  described  in  the  subsequent  paragraphs.  Included  as  external  entities  are  Air 
Force  systems  that  currently  exist  plus  those  that  are  known  to  be  under  study  and/or  development. 
Air  Force  systems  that  emerge  during  the  development  of  the  prototype  AOTS  shall  be  reviewed 
to  determine  if  f  urther  interfaces  are  required.  As  stated  in  Section  3.1.1,  no  electronic  interfaces 
between  the  prototype  AOTS  and  other  computer-based  systems  are  anticipated.  The  methods 
for  exchanging  data  between  the  AOTS  and  external  computer-based  systems,  e.g.,  magnetic  tapes 
or  manual  transfer,  will  be  used. 


AS»CC1tJ*V 


Figure  11.  External  Interface*  with  AOTS 


3.1.5.1.1  United  States  Air  Force  Operational  Measurement  Center  ( USAFOMC, )  Interface. 

An  interface  between  the  AOTS  and  the  USAFOMC  shall  be  established  during  the  prototype 
AOTS  development  phase  (Phase  II)  to  pass  occupational  survey  data  and  analysis  results  to  the 
AOTS  Management  Subsystem  for  use  in  defining  Air  Force  Specialty  (AFS)  tasks,  and  associated 
task  data.  The  data  received  from  the  USAFOMC  shall  include  task  identification  numbers,  task 
statements,  task  factor  analysis  results  and  the  identification  of  support  equipment,  where 
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applicable,  for  all  tasks  reported  during  occupational  surveys  as  performed  in  the  specified  AFSs. 
Further,  this  interface  shall  facilitate  the  AOTS  Management  Subsystem  identifying  to  the 
USAFOMC  Survey  Branch  those  tasks  that  are  found  to  be  performed  by  personnel  assigned 
within  specific  AFSs  that  have  not  been  included  in  the  applicable  occupational  survey  instru¬ 
ments. 


3. 1 .5. 1 .2  Publications  Management  Systems  Interface. 

Interfaces  shall  be  established  during  Phase  II  between  the  AOTS  and  Air  Force  publications 
management  agencies  and  systems  to  acquire  those  regulations,  manuals,  technical  orders,  and 
other  documents  that  prescribe  training  requirements  or  contain  task  performance  and  proficiency 
data.  The  publications  and  documents  shall  be  used  by  AOTS  developers  when  defining  per¬ 
formance  and  training  requirements  within  the  AOTS  Management  Subsystem,  and  for  developing 
behavioral  objectives  and  test  instruments  within  the  AOTS  Evaluation  Subsystem.  An  Air  Force 
Customer  Account  Representative  (CAR)  will  interface  with  the  Publications  Distribution  Office 
(PDO)  at  Bergstrom  AFB  for  ordering  and  receiving  Air  Force  and  Major  Command  publications. 
A  representative  will  also  interface  with  the  Technical  Order  Distribution  Office  (TODO)  at 
Bergstrom. 


3. 1 .5.1 .3  Extension  Course  Institute  (ECI). 

An  interface  with  the  ECI  shall  be  established  during  Phase  II  to  provide  the  AOTS  Management 
Subsystem  with  the  data  required  to  maintain  the  current  availability  status  of  correspondence 
courses.  This  interface  shall  also  facilitate  the  Management  Subsystem  processing  of  corre¬ 
spondence  course  enrollments  for  members  in  upgrade  training  during  the  prototype  AOTS 
demonstration  and  test  phase  (Phase  III).  To  facilitate  the  matching  of  appropriate  correspondence 
courses  to  upgrade  training  requirements,  procedures  shall  be  instituted  whereby  projected  changes 
to  correspondence  course  availability  and  effective  dates  of  changes  are  provided  by  ECI. 


3. 1.5. 1.4  Air  Force  Military  Personnel  Center  (AFMPC)  interface. 

Interfaces  shall  be  established  during  Phase  II  between  the  AOTS  and  the  offices  located  at  the 
AFMPC  having  primary  and/or  collateral  responsibilities  for  training  and  AFS  classification 
policies.  The  current  office  symbol  is  AFMPC/DPMRTC3.  Interfaces  with  the  training  policy 
makers  are  required  for  AOTS  managers  to  receive,  in  sufficient  time  for  planning  and  making 
adjustments,  any  forthcoming  changes  to  policies  that  could  affect  AOTS  management  and 
software  programs.  As  a  minimum,  the  AOTS  Management  Subsystem  will  require  information 
policy  changes  relevant  to  on-the-job  training,  ancillary  training,  additional  duty  training  and 
contingency  task  training. 


3.1 .5.1 .5  Operational  unit  Interfaces  to  Include  Interfaces  with  computer-based  systems  that  support  unit  training 
programs. 

Interfaces  with  operational  units,  to  include  interfaces  with  computer-based  systems  such  as  the 
Core  Automated  Maintenance  System  (CAMS)  and  the  Personnel  Data  System,  shall  be  established 
during  Phase  II.  These  interfaces  are  required  to  collect  information  relevant  to  individual 
qualifications  and  training  course  completions;  to  identify  resources  required  for  training  and 
evaluation;  to  schedule  training  and  evaluation  events;  and  to  provide  reports  relevant  to  training 
progress,  and  AOTS  effectiveness.  During  the  initial  loading  of  data  into  the  prototype  AOTS, 
the  AOTS  Management  Subsystem  shall  collect  from  applicable  operational  units  the  training 
data  currently  maintained  on  training  forms  and  within  computer  data  bases  for  those  personnel 
whose  training  shall  be  managed  within  the  AOTS.  Also  to  be  collected  are  the  nomenclatures 
and  quantities  of  those  training  and  evaluation  resources  which  shall  be  identified  by  the  AOTS, 
and  the  identification  of  contingency  tasks  that  are  controlled  by  these  units.  The  methods  for 
collecting  these  data  and  loading  them  to  the  prototype  AOTS  snail  be  either  by  manual  entry 
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or  by  using  an  electronic  media  (e.g.  magnetic  tape).  During  the  operational  test  phase  for  the 
AOTS,  the  identity  of  training  and  evaluation  events  that  are  not  controlled  by  the  AOTS  shall 
be  sent  to  the  AOTS  Management  Subsystem,  which  shall,  in  turn,  identify  those  personnel 
requiring  the  specified  event  and  record  event  data.  Schedules  for  training  and  evaluation  events 
that  arc  controlled  by  the  Management  Subsystem  shall  be  provided  to  the  applicable  units. 
Formats  for  data  exchange  between  the  AOTS  and  other  systems  shall  be  established  to  facilitate 
exact  interpretation.  During  Phase  III,  training  progress  status  reports  for  individual  trainees 
or  groups  of  trainees  shall  be  provided  by  the  AOTS  Management  Subsystem.  Reports  on  training 
progress  analyses,  training  quality  control,  and  training  system  effectiveness  shall  be  provided 
by  the  AOTS  Evaluation  Subsystem.  The  frequency  and  formats  of  these  reports  shall  be  specified 
by  the  Air  Force.  The  Management  Subsystem  shall  receive  training  data  on  selected  individuals 
by  interfacing  with  the  CAMS.  These  data  shall  be  used  in  scheduling  training  and  evaluation 
events.  The  frequencies  for  providing  these  data,  the  formats  in  which  they  shall  be  provided, 
and  field  sizes  are  specified  in  the  Management  and  Evaluation  Subsystem  Prime  Item  Specifi¬ 
cations  (see  applicable  documents). 


3. 1 .5. 1 .6  Personnel  Data  System  (PDS)  Interface. 

To  facilitate  operation  of  the  AOTS  Management  Subsystem,  an  interface  shall  be  established 
during  Phase  II  between  the  AOTS  and  the  PDS.  The  AOTS  Management  Subsystem  shall  require 
access  to  personnel  data  maintained  within  the  PDS  in  order  to  create  Airman  Training  Records 
(ATRs).  The  types  of  data  required  to  create  an  ATR  include  trainee  names,  social  security 
account  numbers,  grades.  Air  Force  Specialty  Codes  (AFSCs),  Training  Status  Codes  (TSCs), 
Personnel  Accounting  System  (PAS)  codes,  training  course  completions,  education  levels,  Air 
Force  Reading  Abilities  Test  (AFRAT)  scores,  personnel  record  status  codes,  duty  status  codes, 
duty  position  numbers,  office  symbol  codes,  and  Armed  Service  Vocational  Aptitude  Battery 
(ASVAB)  scores.  Changes  to  these  data  shall  be  provided  to  the  AOTS  Management  Subsystem 
as  they  occur  so  that  impacts  on  training  requirements  can  be  determined.  Products  reflecting 
the  unit  manpower  personnel  requirements  shall  be  provided  from  the  PDS  for  use  in  the 
development  of  Position  Task  Requirement  (PTR)  listings.  Training  status  data  used  within  the 
overall  personnel  system  that  shall  reside  within  the  PDS,  shall  be  provided  upon  request  The 
types  of  data  provided  f rom  the  AOTS  to  the  PDS  are  training  progress  status,  such  as  time  in 
training  versus  the  estimated  training  completion  times,  training  course  failures,  qualifications, 
etc.,  that  may  influence  promotion,  demotion,  separation,  classification,  retraining,  or  assignment 
actions.  The  specific  data  to  be  exchanged  between  these  systems  and  the  data  field  sizes  are 
specified  by  the  Management  Subsystem  Prime  Item  Specification  (see  applicable  documents). 
The  data  from  PDS  shall  be  supplied  to  AOTS  on  magnetic  tape.  A  unique  DESIRE  will  be 
required  from  the  Active,  Reserves,  and  Air  National  Guard  PDS  for  each  magnetic  tape. 


3.1 .5.1 .7  Off-line  training  activities  interface. 

Procedures  shall  be  established  during  Phase  II  to  obtain  from  applicable  non-AOTS  training 
agencies;  i.e.,  the  Field  Training  Detachment  (FTD),  the  CBPO-OJT  Unit,  the  Combat  Arms  Range, 
the  Safety  Office,  the  Jud$e  Advocate  Office,  the  Disaster  Preparedness  Office,  the  Drivers 
School,  the  Supply  Custodian  School,  the  Director  of  Administration,  etc.,  that  information 
required  to  identify  personnel  for  training  and  evaluation  events,  and  to  record  progress  and 
completion  status.  Information  relevant  to  event  cancellation  shall  likewise  be  provided.  Training 
and  evaluation  events  scheduled  by  non-AOTS  agencies  shall  be  entered  into  the  AOTS  Man¬ 
agement  Subsystem,  (via  magnetic  tape  media),  and  the  personnel  requiring  the  training  shall  be 
identified  through  an  automated  training  diagnosis  process.  In  some  instances  the  non-AOTS 
agencies  will  have  standard  methods  for  identifying  the  information  that  differs  from  those 
methods  used  in  AOTS,  e.g.,  course  codes  identify  the  same  event  Therefore,  a  method  shall  be 
devised  to  cross-reference  this  information.  Interfaces  shall  be  established  that  enable  the  AOTS 
Management  Subsystem  to  receive  trainee  progress  and  event  completion  data  from  non-AOTS 
sources.  Training  specific  data  to  be  processed,  data  formats,  and  field  sizes  are  specified  in 
the  Prime  Item  Specifications  (see  applicable  documents). 
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3. 1 .5.  t  .8  Core  Automated  Maintenance  System  (CAMS)  Interface t 

The  interface  between  the  AOTS  and  the  CAMS  is  limited  to  extracting  training  data  effecting 
the  selected  AFSs  that  are  part  of  the  prototype  AOTS.  This  interface  shall  be  accomplished  bv 
taking  output  from  CAMS  and  manually  entering  this  information  into  AOTS. 


3.1. 5.2  Internal  interfaces. 


Interfaces  that  shall  be  established  between  AOTS  subsystems  are  functionally  described  in  the 
following  paragraphs. 


3.1 .5.2.1  Management  Subsystem  support  for  the  Evaluation  Subsystem. 

Functions  to  be  performed  by  the  Management  Subsystem  in  support  of  the  Evaluation  Subsystem 
shall  include  the  following: 

a.  Task  identification  and  performance  assessment  requirements  specified  by  the 
Training  Requirements  Management  Component 

b.  Airmen  training  management  program  (e.g.,  progress,  ATR/ITR  composition,  data 
needed  to  evaluate  system  effectiveness) 

c.  Training  scheduling  program 

d.  Trainer  qualifications  and  certified  tasks  for  Training  Quality  Control  evalu¬ 
ation/resource  allocations  and  schedules 


3.1. 5.2.2  Evaluation  Subsystem  support  for  the  Management  Subsystem. 

Evaluation  Subsystem  tasks  shall  include  the  following: 

a.  Identify  evaluation  resources  required  for  evaluation  of  job-site  task  performance 

b.  Develop  behavioral  objectives,  knowledge  test  items,  performance  evaluation 
checklists,  and  oral  test  guides 

c.  Maintain  the  Test  Item  Bank 

d.  Identify  objectives,  items,  and  materials  by  task/subtask  identification 

e.  Randomly  select  appropriate  tasks,  certified  airmen,  and  evaluators  for  Training 
Quality  Control  evaluation  events 

f.  Provide  Training  Quality  Control  evaluation  results 

g.  Deliver  evaluation  materials 

3. 1.5. 2. 3  Management  Subsystem  support  for  the  Computer  Support  Subsystem. 

The  Management  Subsystem  shall  provide  functional  requirements  to  the  Computer  Support 
Subsystem. 


3. 1.5. 2. 4  Computer  Support  Subsystem  support  for  the  Management  Subsystem. 

The  Computer  Support  Subsystem  will  provide  the  following  in  support  of  the  Management 
Subsystem: 
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3. 1.5.2. 4  Computer  Support  Subsystem  support  for  the  Management  Subsystem. 

The  Computer  Support  Subsystem  will  provide  the  following  in  support  of  the  Management 
Subsystem: 

a.  Commercially  available  language  compilers 

b.  Debugging  and  authoring  aids 

c.  Ada  Joint  Program  Office-approved  Ada  programming  compiler 

d.  Hardware  support 

e.  Software  programs  and  editors  to  accomplish  specified  objectives 

3.1. 5.2.5  Management  Subsystem  support  for  the  Personnel  and  Support  Subsystem. 

The  Management  Subsystem  tasks  shall  include  the  following: 

a.  Assistance  in  design  of  training  courses  for  AOTS  personnel 

b.  Specify  personnel  skills  and  training  required 

3.1. 5.2.6  Personnel  and  Support  Subsystem  support  for  the  Management  Subsystem. 

The  Personnel  and  Support  Subsystem  will  provide  the  following  in  support  of  the  Management 
Subsystem: 

a.  Trained  personnel  necessary  to  perform  required  skills 

b.  AOTS  Human  Engineering 

c.  AOTS  Reliability 

3.1. 5.2.7  Management  Subsystem  support  for  the  Training  Development  and  Delivery  Subsystem. 

The  Management  Subsystem  shall  provide  the  following  capabilities  in  support  of  the  Training 
Development  and  Delivery  Subsystem: 

a.  Training  materials  development  requirements 

b.  Individual  trainee  training  requirements 

c.  Resource  requirements  and  schedules 

d.  Task  Training  Requirements 

3.1. 5.2.8  Training  Development  and  Delivery  Subsystem  support  for  the  Management  Subsystem. 

The  tasks  to  be  performed  by  the  Training  Development  and  Delivery  Subsystem  shall  include 
the  following: 

a.  Design  and  develop  training  hierarchies 

b.  Present  embedded  test  items  materials  on-line 

c.  Develop  instructional  strategies 

d.  Collect  trainee  performance  data 
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3.1. 5^.9  Training  Development  and  Delivery  Subsystem  support  for  the  Evaluation  Subsystem. 

The  tasks  to  be  performed  by  the  Training  Development  and  Delivery  Subsystem  in  support  of 
the  Evaluation  Subsystem  shall  include,  but  will  not  be  limited  to,  the  following: 

a.  Assist  in  collection  of  data  to  evaluate  system  effectiveness 

b.  Operationalize  instructional  strategies 

3.1.5.2.10  Evaluation  Subsystem  support  fix  the  Training  Development  and  Delivery  Subsystem. 

The  Evaluation  Subsystem  shall  provide  the  following  capabilities  in  support  of  the  Training 
Development  and  Delivery  Subsystem: 

a.  Effectiveness  data  on  strategy(s)  used  in  evaluation  instruments 

b.  Assist  in  collecting  trainee  task  performance  and  requisite  knowledge  test  data 

c.  Assist  in  ranking  and  selection  of  evaluation  strategies 

d.  Assist  in  data  collection  to  validate  instructional  instruments 

e.  Provide  system  reports  as  required 

3.1 .5.2.1 1  Training  Development  and  Delivery  Subsystem  support  for  the  Computer  Support  Subsystem. 

The  tasks  to  be  performed  by  the  Training  Development  and  Delivery  Subsystem  shall  include, 
but  will  not  necessarily  be  limited  to,  the  following: 

a.  Functional  requirements  to  support  the  accomplishment  of  Training  Development 
objectives 

b.  Functional  requirements  to  support  the  accomplishment  of  Training  Delivery 
objectives 


For  purposes  of  the  prototype  AOTS,  the  CAI  capability  from  the  existing  Government  owned 
Instructional  Support  System  (ISS)  shall  be  used  to  the  maximum  extent  possible. 


3.1.5.2.12  Computer  Support  Subsystem  support  for  the  Training  Development  and  Delivery  Subsystem. 

The  Computer  Support  Subsystem  shall  provide  the  following  in  support  of  the  Training 
Development  and  Delivery  Subsystem: 

a.  Commercially  available  language  compilers 

b.  Debugging  and  authoring  aids 

c.  Ada 

d.  Terminal  and  printer  support 

e.  Software  programs  and  editors  required  to  accomplish  specified  objectives 
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3.1 .5.2.1 3  Training  Development  and  Delivery  Subsystem  support  for  the  Personnel  and  Support  Subsystem. 

The  tasks  to  be  performed  by  the  Training  Development  and  Delivery  Subsystem  shall  include, 
but  will  not  be  limited  to,  the  following: 

a.  Assist  in  the  design  and  development  of  necessary  training  courses 

b.  Specify  personnel  skills  required 

3.1.5.2.14  Personnel  and  Support  Subsystem  support  for  the  Training  Development  and  Delivery  Subsystem. 

The  Personnel  and  Support  Subsystem  shall  provide  the  following  in  support  of  the  Training 
Development  and  Delivery  Subsystem: 

a.  Trained  personnel  necessary  to  perform  required  skills 

b.  AOTS  Human  Engineering 

3.1.5.2.15  Evaluation  Subsystem  support  for  the  Computer  Support  Subsystem. 

Evaluation  Subsystem  capabilities  shall  include,  but  not  be  limited  to,  functional  requirements 
to  support  completion  of  Evaluation  Subsystem  objectives.  These  objectives  are: 

a.  To  plan,  develop,  validate,  and  maintain  evaluation  instruments 

b.  To  administer  evaluation  instruments  and  collect  performance  evaluation  results 

c.  To  provide  quality  control  evaluations  for  assessment  of  training  effectiveness 

d.  To  provide  unit  training  and  system  effectiveness  reports 

3.1.5.2.16  Computer  Support  Subsystem  support  for  the  Evaluation  Subsystem. 

The  Computer  Support  Subsystem  shall  provide  the  following  in  support  of  the  Evaluation 
Subsystem: 

a.  Commercially  available  language  compilers 

b.  Debugging  and  authoring  aids 

c.  Ada 

d.  Hardware  support 

e.  Software  programs  and  editors  to  accomplish  specified  objectives. 

3.1.5.2.17  Evaluation  Subsystem  support  tor  the  Personnel  and  Support  Subsystem. 

Evaluation  Subsystem  tasks  shall  include,  but  not  be  limited  to,  the  following: 

a.  Specify  personnel  skills  required  for  performance  of  Evaluation  Subsystem  objectives 
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3.1.5.2.18  Personnel  and  Support  Subsystem  support  for  the  Evaluation  Subsystem. 

The  Personnel  and  Support  Subsystem  will  provide  the  follow.,  g  in  support  of  the  Evaluation 
Subsystem: 

a.  Train  personnel  necessary  to  perform  tasks  required  by  Evaluation  Subsystem 

b.  AOTS  Human  Engineering 


3.1.5.2.19  Personnel  and  Support  Subsystem  support  for  the  Computer  Support  Subsystem. 

The  tasks  to  be  performed  by  the  Personnel  and  Support  Subsystem  in  support  of  the  Computer 
Support  Subsystem  shall  include,  but  will  not  be  limited  to,  the  following: 

a.  Train  personnel  to  perform  required  AOTS  tasks 

b.  Hardware  and  Software  Reliability  reports 

e.  Maintenance  logistics  support 

f.  Monitor  Human  Engineering  requirements  in  the  design  of  AOTS. 


3.1.5.2.20  Computer  Support  Subsystem  support  for  the  Personnel  and  Support  Subsystem. 

The  Computer  Support  Subsystem  shall  provide  the  following  in  support  of  the  Personnel  and 
Support  Subsystem: 

a.  Hardware  logistics/maintenance  requirements 

b.  Specific  personnel  skills  needed  to  perform  necessary  jobs. 


3.1.6  Government-furnished  property  list. 

3. 1.6.1  Government-furnished  computer  programs. 

Government-furnished  computer  programs  which  AOTS  may  utilize  as  is  or  which  may  be  modified 
to  meet  the  requirements  of  this  specification  will  include  Instructional  Support  System  (1SS), 
General  Imagery  Intelligence  Training  System  (GIITS),  or  any  others  that  may  be  deemed 
applicable,  various  elements  of  the  ISS  software,  (e.g.  system  support,  virtual  machine,  CAI,  test 
analysis,  etc.,  software  packages),  and  the  GUTS,  (e.g.  the  file  management  interface  with  the 
computer),  are  available  for  use  in  AOTS. 


3. 1.6.2  Other  government-furnished  property. 

The  Government  will  furnish  power,  some  of  the  communication  cabling  and  connections,  and 
facilities  as  specified.  The  Government  shall  also  provide  Zenith  Z-248  personal  computers, 
associated  printers,  digitizer  tablets  (small)  and  software,  and  use  of  a  Digital  Equipment  Cor 
poration  VAX  8650  computer  system.  The  Government  shall  also  provide  Digital  Equipment 
Corporation  VT100  computer  terminals  and  Tektronix  4105  color  computer  terminals. 
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3.1. 6.2.1  Power. 


The  government  wilt  furnish  adequate  electrical  power.  The  government  will  interface  with  the 
electrical  power  provider  to  resolve  any  power  problems.  The  electrical  power  will  meet  normal 
standards,  including  voltages  within  the  tolerance  of  +15%  or  -10%  of  nominal  levels,  frequency 
variations  less  than  +/-  1  hertz,  and  no  surges  or  brown-outs. 

3.1. 6.2.1. 1  Connecting  devices  power  requirements. 

The  government  will  provide  120  volt,  60  hertz,  single-phase  power  to  each  connecting  device 
by  way  of  a  wall-mount  receptacle.  The  power  will  be  protected  from  electrical  surges  or 
brown-outs.  Each  connecting  device,  except  the  terminals  and  mark  readers,  will  require  one 
wall  outlet  with  a  20-amp  receptacle  supplying  a  maximum  of  3  amps.  The  terminarcontroller 
and  monitor  will  each  require  one  receptacle  with  a  total  load  of  less  than  3  amps  each.  All 
power  provided  to  connecting  devices  should  be  on  independent  circuits.  If  that  is  not  possible, 
circuits  will  be  provided  which  do  not  have  large  electrical  noise-inducing  components  such  as 
calculators,  typewriters,  or  copiers. 


3. 1 .6.2.2  Communication  cabling  and  connections. 

The  government  will  provide  building-to-building  communications  lines.  The  lines  will  run  from 
government-provided  terminal  blocks  in  the  building  that  contains  the  computer  component  to 
government-provided  terminal  blocks  in  the  buildings  that  contain  the  connecting  devices.  Each 
connecting  device  or  group  of  connecting  devices,  where  multiplexed,  will  require  a  four-wire 
circuit.  Adequate  backup  four-wire  circuits  should  be  provided  in  the  event  of  communication-line 
failure.  Full  requirements  shall  be  a  part  of  the  Site  Preparation  and  Installation  Plan(s). 


3.1. 6.2.3  Facilities. 


The  government  will  provide  facilities  to  meet  the  following  requirements. 

3.1. 6.2.3. 1  Computer  component  failures. 

Should  a  dedicated  computer  system  be  required  (lack  of  availability  or  capacity  of  the  Gov¬ 
ernment  furnished  DEC  VAX  8650  computer  system  that  is  located  at  AFHRL/SC  on  Brooks 
AFB,  TX.),  then  the  computer  component  will  operate  under  the  following  environmental 
constraints.  The  government  will  provide  facilities  that  meet  and  maintain  the  following  con¬ 
ditions. 


3.1.6.2.3.1.1  Computer  component  environment. 

The  following  limits  indicate  the  range  of  environmental  factors  for  the  computer  room  envi¬ 
ronment.  Out-of -range  conditions  beyond  the  maximum  limits  may  cause  the  system  to  be  shut 
down,  or  may  damage  the  equipment.  The  air-conditioning  system  should  provide  the  range  of 
temperature  and  humidity  indicated  under  the  recommended  limits. 

Maximum  limits 


a.  65-85  degrees  Fahrenheit 

b.  Maximum  rate  of  change  of  temperature  5.5  degrees  Fahrenheit  per  hour 

c.  20-80  percent  relative  humidity,  noncondensing 

d.  Maximum  rate  of  change  of  relative  humidity  6  percent  per  hour 
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e.  Low  dust,  filtered  air 

f.  Clean  environment 

Recommended  Limits 

a.  65-75  degrees  Fahrenheit 

b.  45-55  percent  relative  humidity,  noncondensing 

c.  Nearly  dust  free 


3.1.6.2.3.1.2  Computer  facility  floor. 

A  raised  computer  facility  floor  will  be  provided  to  make  room  for  power,  control,  and  com¬ 
munication  cables.  The  floor  should  be  raised  at  least  9  inches  above  the  building  floor  level. 
A  ramp  at  least  4  feet  long  will  lead  from  the  building  floor  to  the  raised  floor.  Both  the  ramp 
and  the  computer  facility  floor  will  be  capable  of  supporting  an  average  load  of  200  Ib./sq.ft. 
and  will  be  capable  of  supporting  a  concentrated  load  of  1000  lb./sq.in.  Floor  vibrations  of  any 
magnitude  will  not  be  acceptable  while  the  computer  is  operational. 


3.1.6.2.3.1.3  Computer  facility  doorways. 

To  provide  entry  and  removal  of  computer  hardware  at  the  computer  facility,  doorways  along 
the  path  of  entrance  will  prov:de  at  least  a  36-inch  opening.  A  6-foot  deep  area  will  be  provided 
on  each  side  of  doorways  -vh-re  hardware  will  be  turned. 


3.1.6.2.3.1.4  Fire  protection. 

The  government  will  provide  a  method  of  detecting  and  extinguishing  any  fire  in  the  computer 
component  facility.  The  extinguishing  method  will  not  cause  further  damage  to  the  computer 
component.  A  Halon  system  is  recommended.  The  system  will  be  automatically  initiated  and 
will  provide  ample  warning  to  personnel.  The  fire-detection  scheme  will  also  shut  down  the  air 
conditioning  system.  The  power  system  component  will  have  an  interconnect  available  for  use 
by  the  government  in  shutting  off  the  air-conditioning  system  in  the  event  that  fire  is  detected. 
If  the  government  does  not  choose  to  use  the  interconnect,  the  government  will  provide  a  method 
for  shutting  down  the  air-conditioning  system. 


3.1.6.2.3.1.5  Water  protection. 

The  government  will  provide  an  alarm  system  capable  of  detecting  the  presence  of  water  or  other 
liquids  seeping  into  the  computer  component  facility.  That  alarm  system  also  will  provide  a 
normally  closed  or  normally  open  breaker  for  sensing  by  the  power  system  component  to  provide 
shutdown  of  the  computer  system. 


3.1.6.2.3.2  Connecting  devices. 

The  government  will  provide  an  office-quality  environment  for  the  connecting  devices  that  wil! 
meet  the  following  constraints: 

a.  65-90  degrees  Fahrenheit 

b.  20-80  percent  relative  humidity,  noncondensing. 
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In  addition,  the  connecting  devices  will  not  be  exposed  to  excessive  dirt  or  dust  nor  will  they  be 
exposed  to  spilled  substances  or  to  grease.  The  connecting  devices  will  not  be  placed  in  an  area 
where  eating  and  drinking  are  permitted. 


3.1.7  Operational  and  organizational  concepts. 
3.1. 7.1  Basic  performance  parameters^ 


a.  Operational  factors 


1.  OJT  program  structure/policy  considerations 

(a)  The  AOTS  shall  comply  with  Air  Force  training  policies  unless  deviations 
from  governing  directives  have  been  approved.  It  shall  be  supportive  of 
current  classification  structures  and  be  capable  of  adapting  to  AFSC 
conversions. 


(b)  To  the  greatest  extent  possible,  the  AOTS  shall  provide  automated  means 
of  identifying  duty  position  performance  requirements,  identifying 
individual  and  group  training  requirements,  scheduling  training  events, 
developing  evaluation  instruments,  evaluating  training  and  system  per¬ 
formance,  and  developing  and  delivering  training.  Major  goals  shall  be 
(1)  where  possible,  to  reduce  the  workload  experienced  by  employing  the 
current  OJT  system;  (2)  where  feasible,  to  streamline  methods  of  pro¬ 
ducing,  delivering,  recording,  and  evaluating  training;  and,  (3)  where 
appropriate,  to  introduce  new  technologies  into  the  workcenter  OJT  arena 
that  have  not  heretofore  been  available. 


(c)  From  the  results  of  the  operation,  testf  and  evaluation  of  AOTS,  sug¬ 
gestions  shall  be  provided  on  configuration(s)  and  concepts  for  managing 
the  system  when  it  is  turned  over  to  the  Air  Force  or  for  Full  Scale 
Engineering  Development  for  future  Air  Force  wide  deployment. 

2.  Environmental  considerations 


(a)  The  AOTS  peripherals  shall  be  capable  of  operating  within  areas  having 
high  concentrations  of  dust,  grease,  etc.,  particularly  in  workcenters 
supporting  aircraft  maintenance.  Peripherals  that  have  alternative  models 
that  can  operate  within  these  types  of  environments  shall  be  given 
preference. 

(b)  The  AOTS  shall  be  user-friendly.  It  shall  allow  airmen  with  differing 
levels  of  experience  to  operate  the  system  with  a  minimum  of  training. 
Menus  and  prompts  shall  be  used  to  promote  easy  understanding  of  what 
is  required  from  the  user  as  input  and  the  output  or  results  being  displayed. 
Experienced  users  shall  have  options  to  abbreviate  the  prompts  and  menus 
if  desired. 


b.  Organizational  factors.  In  that  each  organization  may  be  structured  differently  and 
that  shift  work  may  be  an  integral  element,  the  AOT$  shall  be  capable  of  operating 
24  hours  per  day. 
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3.1 .7.2  AOTS  relationships  with  other  Systems^ 

Interfaces  and  relationships  with  systems  already  operational  in  the  Air  Force  are  defined  in 
3.1.5.  As  existing  systems  are  modified  or  changed,  the  Air  Force  PM  will  ensure  that  the 
contractor  is  aware  of  these  changes  and  is  furnished  sufficient  documentation  to  effect  any 
modifications  to  the  AOTS  that  may  be  necessary.  These  changes  shall  be  incorporated  as 
necessary.  The  mission  of  the  AOTS  in  relationship  to  other  AF  systems  shall  be  unique  in  its 
purpose  and  supportive  of  other  training  systems  in  the  Air  Force.  Currently,  there  is  no  Air 
Force-standardized  mechanized  support  for  OJT  except  for  the  Personnel  Data  System  which 
records  only  a  few  "bits*  of  data  pertaining  to  OJT  such  as  Primary,  Secondary,  Duty  and  Control 
AFSCs,  dates  entered  training  or  retraining,  date  upgraded,  and  Training  Status  Code.  No  system 
manages  the  determination  of  task  performance  requirements,  the  selection  and  prioritization  of 
training  requirements,  and  the  systematic  scheduling  of  personnel  to  training  or  evaluation  events. 


3. 1.7.3  Anticipated  deployment 

a.  Prototype  Application 

(1)  The  prototype  AOTS  shall  be  designed  to  accommodate  an  annual  training  load 
of  approximately  800  airmen  and  direct  users  of  the  system  (including 
supervisors,  trainers,  trainees,  and  the  1ST).  It  shall  provide  the  mechanisms 
by  which  all  aspects  of  job-site  training  shall  be  managed  in  each  of  the 
designated  workcenters  from  the  67th  Tactical  Reconnaissance  Wing  (67th 
TRW),  the  924th  Tactical  Fighter  Group  (924  TFG-AFRES)  at  Bergstrom  AFB, 
Texas  and  the  147th  Fighter  Interceptor  Group  of  the  Air  National  Guard 
located  at  Ellington  ANGB,  Texas.  Test  Air  Force  Specialties  designated  by 
the  AF  are: 


(a)  454X0A  Aerospace  Propulsion 

(b)  452X4D/M  Tactical  Aircraft  Maintenance 

(c)  732X0  Personnel 

(d)  811X0  Security 

(e)  811X2  Law  Enforcement 

(2)  Following  completion  of  the  system  test  and  evaluation  of  the  prototype,  the 
contractor  shall  deliver  the  prototype  AOTS  to  AFHRL  at  the  development 
site. 


3.2  Characteristics. 

3.2.1  Performance  characteristics. 

The  system  performance  shall  be  determined  during  each  phase  of  the  program. 

3.2.1. 1  Response  performance  goals. 

Response  performance  goals  for  the  AOTS  development  system  for  each  phase,  operating  under 
normal  load,  shall  be  (average): 

a.  Phase  II  -  2  seconds  (worst  case) 

b.  Phase  III  -  2.5  seconds  (worst  case) 

c.  Master  Task  List  Search  -  Phase  III  •  1  minute  maximum 

d.  Compile  rate  during  software  development  -  600  LPM  compile  a .... 
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3.2. 1.2  Computer  components. 

The  AOTS  Computer  Component  Subsystem  specification  70S647400  consists  of  the  following 
components: 


a.  Computer  System  Component 

b.  Terminal  Component 

c.  Optical  Mark  Reader  Component 

d.  Printer  Component 

e.  Digitizer  Tablet  Component 

f.  Terminal  Component  with  Interactive  Video  Disk  (IVD)  capability 
3.2. 1.2.1  Computer  component  goal  specification. 

The  specific  characteristics  that  will  be  used  as  goal  specifications  for  the  Computer  System 
Component  are: 

a.  Computational  Capacity  -  6  MIPS  PHASE  II 

b.  Computational  Capacity  -  8  MIPS  PHASE  III 

c.  Computational  Capacity  -  8.5  MIPS  PHASE  IV 

d.  Disk  Storage  subsystem 

1.  1.3  GigaBytes  (GB)  Disk  Storage  -  PHASE  II  &  III 

2.  1.7  GB  Disk  Storage  -  PHASE  IV 

e.  Computer  Memory  subsystem 

1.  Minimum  of  32  MB  main  memory  Phase  II 

2.  Minimum  of  64  MB  main  memory  Phase  IV 

f.  Magnetic  Tape  Subsystem 

1.  Magnetic  Tape  Backup  Subsystem 

g.  Line  Printer  -  445  LPM  96  Character  Set 

h.  Communications  subsystem 

1.  RS232C  Communications  capability  (serial) 

(a)  64  Ports  -  Phase  II 

(b)  124  Ports  -  Phase  III  &  IV 

i.  Virtual  Memory  Capability 

j.  Floating  Point  Accelerator 

k.  Cache  Memory  ■  Minimum  8  KiloBytes  (KB) 

l.  Vector  Priority  Interrupt  Structure 

m.  Real  Time  Clock  and  Interval  Timer 

(a)  Real  Time  clock  minimum  resolution  of  .001  second 

(b)  Interval  Timer  minimum  resolution  of  1  millisecond 

n.  I/O  Bandwidth  of  20  MB/seconds  Minimum 

o.  Virtual  Memory  Disk  Based  Operating  System 

p.  Validated  Ada  Compiler 
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q.  Support  System  Software  (e.g.,  editors,  debuggers.  Source  Code 
Control  System) 

r.  MTBF  600  hours  minimum  (goal) 

s.  MTTR  1.5  hours  Maximum  (goal) 

t.  Offer  Full  Service  Maintenance  Contracts 

1.  Hardware 

2.  Software 

3.2. 1.2.2  Terminals. 

The  terminals  of  the  AOTS  system  shall  use  Personal  Computers  only  for  ease  of  expandability. 
The  terminal  specification  to  be  used  as  goals  for  the  AOTS  development  program  are: 

a.  80286  Microprocessor  at  6  MHz  (minimum)  or  equal 

b.  MS-DOS  3.1  or  higher  operating  system  or  equal 

c.  IBM  AT  compatible  or  equal 

d.  512  KB  minimum  main  memory  expandable  to  6  MB  minimum 

e.  Serial  interface 

f.  Parallel  printer  port 

g.  QWERTY  Keyboard,  AT  Compatible  or  equal 

(1) .  Detached 

(2) .  10  function  switches,  numeric  keypad  and  cursor  control 

h.  360KB  floppy  disk  (minimum) 

i.  20.0  MB  hard  disk 

j.  MTBF  2000  hours 

k.  MTTR  0.5  hour 

l.  Graphics/Text  controller 

m.  Interactive  video  (optional) 

(1) .  Support  genlock 

(2) .  Graphics/Text  overlay  capable 

(3) .  Standard  525  commercial  color  TV  compatible 

(4) .  Acceptable  video  sources: 

(a) .  Video  Disk 

(b) .  Video  Tape 

n.  Monitor  (Graphics/Text  color) 

(1) .  Medium  resolution  (640  X  480) 

(2) .  12  to  14  Inch  (diagonal) 

(3) .  Color 

(4) .  Minimal  flicker  (medium  to  long  persistence  phosphor) 

(5) .  9300  degree  Kelvin  color  temperature 

(6) .  Interlaced  and  non-interlaced  switchable  (interlaced  at  ruin’irunv' 

(7) .  Tilt/Swivel  (or  equivalent  means  of  achieving  this) 
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(8). 

Low  glare  screen  (preferred) 

(9). 

MTBF  of  15,000  hours  minimum 

(10). 

MTTR  of  0.5  hour 

(11). 

Analog  or  TTL  RGB  input 

р.  Monitor  (video/graphics/text  color)  -  additional  characteristics 

(1).  650 0  Degree  Kelvin  Color  Temperature 

3.2. 1 .2.3  Optical  mark  sense  reader. 

The  optical  mark  sense  reader  shall  have  the  following  general  characteristics: 

a.  Optical  technology  used. 

b.  12  inches  per  second  scanning  rate  minimum 

с.  8.5  X  11  inch  paper  size  minimum 

d.  Bubble  matrix  80  X  25  X  2  minimum 

e.  Optional  printer/terminal  interface 

3.2. 1.2.4  Printers. 

The  characteristics  of  the  printers  are  for  both  color  and  monochrome.  The  specifications  are: 

a.  Printer  -  Color 

(1) .  Minimum  of  4  colors  (preferred  is  8  colors) 

(2) .  Support  graphics  (for  screen  dumps  and  general  graphics/plotting  capa¬ 

bility) 

(3) .  Bi-Directional  printing  (optional) 

(4) .  Multi-Densities  printing  (draft,  intermediate,  high  quality) 

(5) .  Printing  Speeds: 

(a) .  High  -  100  character  per  second  (CPS) 

(b) .  Low  -  35  CPS 

(6) .  Dot  matrix,  100  dots  per  inch  minimum 

(7) .  MTBF  2000  hours  minimum 

(8) .  MTTR  0.5  hour 

(9) .  Regular  bond  paper  capability  preferred  (e.g.  201b  standard  8.5  x  11  inch 

bond  paper) 

b.  Printer  -  Monochrome 

( 1 ) .  Dot  matrix 

(2) .  Impact  or  laser  technology 

(3) .  Minimum  of  300  CPS  printing  speed  (draft  quality) 

(4) .  Support  graphics 

(5) .  Variable  pitches  6  -  16.6 

(6) .  Resolution  of  240  dots  per  inch  minimum 

(7) .  Draft  resolution  of  60  dots  per  inch  is  acceptable 
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(8) .  Bond  paper  single  sheet  feed  capable 

(9) .  Fanfold  paper  capable  with  tractors  (for  impact  printer  type) 

(10) .  Alternate  fonts  (Minimum  of  4) 

(11) .  Bi-Directional  printing  (optional) 

(12) .  MTBF  of  5000  hours  minimum 

(13) .  MTTR  of  0.5  hour 

(14) .  Noise  level  not  to  exceed  55  db  for  sensitive  areas 

3.2. 1.2.5  Digitizer  tablet. 

The  digitizer  tablet  (or  pad)  characteristics  are: 

a.  11X11  inch  minimum  digitizing  area 

b.  20  X  20  inch  digitizing  area  required  for  large  drawing  digitizing 

c.  Resolution  of  .005" 

d.  Accuracy  of  .005" 

e.  Cursor  puck  with  minimum  of  3  control  buttons 

3.2. 1 .2.6  Specifications  for  all  Items. 

The  following  characteristics  act  as  specifications  for  all  items  covered  in  this  section: 

a.  All  items  must  be  safe  to  operate  and  handle 

b.  Easy  to  transport 

c.  Meet  office  environment  (unless  otherwise  stated) 

d.  Operating  temperature  range: 

(1) .  Computer  system:  65  to  75  degrees  Fahrenheit 

(2) .  All  other  components:  55  to  90  degrees  Fahrenheit 

e.  20  to  80  percent  relative  humidity  (non  condensing) 

3.2.2  Physical  characteristics. 

No  special  constraints  exist  for  the  prototype  AOTS. 

3.2.3  Reliability. 

3.2.3. 1  Hardware  and  software  reliability. 

A  tailored  reliability  program  shall  be  developed  and  maintained  in  accordance  with  MIL-STD-785. 
It  shall  identif  y  and  include  all  program  management  tasks  in  a  manner  that  permits  the  economical 
achievement  of  overall  program  objectives.  Reliability  tasks  shall  be  used  to  evaluate  system 
reliability,  identify  problems,  and  recommend  solutions.  Quantitative  reliability  requirements 
shall  be  incorporated  into  the  Reliability  Plan.  Specific  reliability  data  shown  in  Table  1  sha 
be  used  as  a  goal  for  Phase  II  and  Phase  III  of  the  AOTS  program.  (Since  Fl^se  IV  nsc.  the 
same  system,  the  data  will  also  apply  to  this  phase.)  Continually  gathered  data  wi"  he  used  to 
update  this  specification.  With  deployment  of  AOTS  Air  Force  wide,  the  icliabiii:’  date  will 
be  mandatory.  The  following  techniques  shall  be  used  to  achieve  required  ’  dr  r 
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a.  Use  various  diagnostic  routines 

b.  Use  of  automatic  error  checking  (built  into  hardware) 

c.  Modular  design  (board  level  repair/replace) 

d.  Component  failure  messages  to  operators  (via  reports,  logs,  direct  displayed  messages, 
etc.) 

e.  Identification  of  impending  failures  (i.e.,  graceful  degradation) 

Mission  reliability  prediction  will  be  performed  in  accordance  with  MIL-STD-785B,  paragraph 
50.2.2.3.  Table  1  contains  the  reliability  goals  requirements. 

Table  1.  Reliability  Goal  Requirements  for  Phase  II  and  Phase  III  of  the  AOTS  Program 


Parameter 

Threshold 

Estimate 

MTBF 

250  Hours 

400  Hours 

MTBDE 

400  Hours 

TBD 

Mission  Reliability 

96.5% 

97.5% 

Table  2  provides  reliability  and  maintainability  quantitative  definitions. 


THE  REST  OF  THIS  PAGE  LEFT  BLANK.  INTENTIONALLY 
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Tabl*  2.  Definitions  of  Reliability  and  Maintainability  Parameters 


Parameter 

Definition 

Mean  Time  Between  Downing  Events 
(MTBDE) 

Based  on  no  greater  than  five  terminals  being 
down  at  one  time. 

Mean  Time  Between  Failure  (MTBF) 

The  mean  time  between  relevant  failures 
excluding  preventive  maintenance. 

Mission  Reliability 

The  ability  of  an  item  to  perform  its  required 
functions  for  the  duration  of  a  specified 
mission  profile.  (Includes  equipment  and 
functional  redundancies,  BITE,  and  support 
equipment.) 

Mean  Preventive  Time  (Mpt) 

The  mean  time  to  perform  preventive  main* 
tenance  on  the  system. 

Mean  Time  To  Repair  (MTTR) 

The  mean  time  to  perform  corrective  main¬ 
tenance  on  the  system. 

Mean  Time  Between  Maintenance  (MTBM) 

The  mean  time  between  maintenance 
actions-either  corrective  or  preventive. 

Administrative  Delays  (Adt) 

The  mean  time  involved  in  supporting  the 
maintenance  activity  (e.g.,  Paperwork,  etc.). 

Logistics  Delay  Time  (Ldt) 

The  mean  time  involved  in  providing  items  to 
the  maintenance  activity  (e.g.,  the  time  to 
deliver  parts,  etc.). 

Mean  Maintenance  Time  (M) 

Mpt  +  MTTR 

Mean  Maintenance  Down  Time  (MDT) 

(MTTR  +  Mpt)  +  (Adt  +  Ldt) 

Mean  Time  to  Restore  (MTR) 

MTTR  +  mean  time  to  restart  program 

Operational  Availability  (Ao) 

MTBM 

A 9  MTBM* MDT 

(Adt  -  4 

Ldt  -  0) 

Software  standards,  testing,  and  quality  assurance  are  pertinent  to  software  reliability.  An  AOTS 
Software  Development  Plan  shall  be  maintained  to  describe  these  methodologies.  Accepted 
computer  programming  standards,  in  accordance  with  the  AOTS  Software  Development  Plan  and 
Computer  Programming  Standards  Manual,  shall  be  strictly  adhered  to  in  software  development. 
Software  testing  shall  be  performed  on  the  unit,  program,  and  system  level.  This  testing  shall 
be  in  accordance  with  the  AOTS  Software  Development  Plan  and  Computer  Software  Test  Plan. 
A  software  quality  assurance  plan  shall  ensure  that  reliable  software,  which  fulfills  AOTS 
requirements,  is  maintained  throughout  the  software  life  cycle.  In  addition  to  these  methodologies, 
the  following  techniques  shall  be  used  to  achieve  required  levels  of  reliability  for  software: 

a.  Data  validity  checking 

b.  Hardware  and  software  error  recognition  (exception  handling) 

c.  Program  failure  messages  to  operators 

d.  Avoidance  of  failed  hardware  (degraded  mode  of  operation). 
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3. 2.3. 2  System  reliability. 

AOTS  shall  be  designed  to  ensure  system  reliability.  System  reliability,  aside  from  hardware 
and  software  concerns,  shall  be  tested  at  the  user  interface  level  during  the  development  and 
implementation  of  AOTS.  System  reliability  issues  center  around  the  concern  users  have  for  the 
consistency  and  accuracy  of  output  after  input  has  been  processed.  The  users  will  also  be 
concerned  about  how  easily  system  corrections  can  be  made  if  processing  problems  are  discovered. 
Users  of  AOTS  will  review  trainee  and  system  status  data  to  determine  If  the  system,  as  designed, 
is  working  consistently  and  accurately  (i.e.,  reliably).  For  example,  if  trainees  are  failing  Quality 
Control  evaluations  at  unacceptable  rates,  then  it  is  apparent  that  AOTS  is  not  functioning 
properly  (i.e.,  it  is  not  consistent:  trainees  are  being  certified  but  cannot  perform  their  tasks). 
This  information  triggers  the  problem-solving  process.  Maintainability  allows  the  user  to  trace 
effects  to  causes  for  this  phenomenon  (see  3.2.4  below). 


3.2.4  Maintainability. 


The  following  factors  are  an  AOTS  requirement  in  the  design  evolution  of  the  AOTS  maintenance 
concept  with  MIL-STD-470,  MIL-STD-421,  and  MIL-STD-1472  used  as  guides: 


a.  Use  of  built-in  tests  for  both  rapid  detection,  isolation  and  correction  of  failures. 

b.  Use  of  functional  reverification  to  ensure  proper  operation. 

c.  Use  of  standard  tools  and  equipment  to  maximize  interchangeability;  or 

d.  Exchange  of  failed  items. 

e.  Use  of  modular  design  to  enhance  diagnostic  capabilities. 


Also  sec  Table  2  for  some  of  the  information  on  maintainability  that  is  normally  associated  or 
works  in  conjunction  with  reliability. 


3.2.4. 1  Hardware  malfunctions. 


Hardware  system  malfunctions  shall  be  repaired  on-line  by  the  removal  and  replacement  of 
modules,  assemblies,  or  complete  terminals  as  appropriate.  BIT  and  computer  diagnostics  shall 
be  employed  to  the  maximum  possible  extent  to  facilitate  identification  and  replacement  of  failed 
modules  or  assemblies.  The  first-line  maintenance  shall  also  include  all  scheduled  preventive 
maintenance  tasks.  Second-line  maintenance,  which  includes  repair  by  replacement  of  faulty 
cards  or  circuit  boards,  shall  be  accomplished  on-site  for  the  host  computer  system.  Second-line 
maintenance  on  terminals  may  be  effected  off-site  by  the  designated  repair  authority.  All 
third-line  maintenance,  which  includes  component  removal  and  replacement,  shall  be  effected 
off-site  by  the  designated  repair  authority. 


3.2.4.2  Maintainability  requirements. 

Maintainability  requirements  shall  not  be  construed  so  as  to  impose  design  constraints  on  off- 
the-shelf  hardware.  Records  shall  be  maintained  on/for: 

a.  A  tracking  system 

b.  Location  of  equipment 

c.  Quantity  of  equipment  at  a  location 

The  specifications  in  Table  3  for  the  maintainability  of  the  system,  its  components,  and  subsystems 
shall  apply. 
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Table  3.  Maintainability  Specifications 


Parameter 

Threshold 

MTTR  on  the  System 

90%  upper  limit  to  repair 

1.5  Manhours 

2.5  Manhours 

Mean-Time-To-Restore  the  system 

90%  upper  limit  to  restore 

0.5  Manhours 

1.0  Manhours 

Mean-Time-To-Remove  and  Replace 

90%  upper  limit  to  remove  and  replace 

0.4  Manhours 

1.25  Manhours 

Direct  maintenance  manhours  per  equipment 
operating  hour 

0.05  Hours 

Response  time  to  site  of  failure 

4.0  Hours 

Principal  Period  of  Maintenance 

16  hours  /  day 

7  days  per  week 

Off  line  maintenance  average  MTTR 

90%  upper  limit  to  repair 

1.5  Manhours 

6.0  Manhours 

Mean  preventive  maintenance  time 

3  Hours 

Mean  Maintenance  Time 

3.4  Hours 

Logistic  delay  time 

4  Hours  (90%tile) 

Maximum  corrective  maintenance  time 

8  hours 

Administrative  delay  time 

2  hours 

Maintenance  downtime 

7.4  hours 

Maintenance  manhours/month 

6  hr/mo. 

Frequency  of  Preventive  Maintenance 

1  time  per  month  per  workcenter 

3  2.4.3  Maintenance  techniques. 

Maintenance  on  the  system  can  be  accomplished  using  the  following  techniques: 

a.  Maintenance  contracts  with  vendors 

b.  Time  and  material  billing  for  repair  of  equipment 

c.  Prime  contractor’s  maintenance  personnel 

d.  Other  (must  be  identified  in  the  maintance  plan) 

3.2.4.4  Software  maintainability. 

The  computer  system  component  of  the  AOTS  system  must  have  periodic  software  backup 
performed.  The  following  specification  applies: 

a.  Weekly  complete  backups  of  all  data  stored  on  disk  drives 

b.  Daily  backup  on  changed  files  only 
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The  previous  weekly  backups  must  be  kept  in  a  separate  location  from  the  computer  location. 
All  software  master  copies  of  vendor  supplied  software  must  be  backed  up  and  kept  at  a  different 
location.  Software  maintenance  shall  also  include  the  following  functions: 

a.  Installation  of  new  software  or  updates 

b.  Restoration  of  all  data  should  a  failure  in  the  disk  subsystem  occur  or  a  system 
"crash”  occur  resulting  in  loss  of  all  data  on  a  disk  subsystem 


3.2.4.5  System  maintainability. 

AOTS  shall  be  designed  to  ensure  system  maintainability.  System  maintainability,  with  the 
exception  of  software  and  hardware,  shall  be  tested  at  the  user  interface  level  during  the 
development  and  implementation  of  the  prototype  AOTS.  A  key  aspect  of  maintainability  is 
traceability.  For  hardware  and  software,  documentation  allows  one  to  trace  from  effects  back 
to  causes.  Similarly,  AOTS  products  shall  be  associated  with  one  another  by  referential,  or 
linking,  identifiers.  For  example,  task  information  shall  be  linked  to  behavioral  objectives, 
behavioral  objectives  to  test  items,  test  items  to  evaluation  instruments,  etc.  These  connections 
provide  a  context  for  any  one  element  and  can  help  those  responsible  for  system  maintainability 
to  adapt  the  system  to  changing  conditions  or  to  isolate  deficiencies  that  result  in  unreliable 
outputs.  As  system  deficiencies  are  detected  through  the  use  of  system  reports,  problem  areas 
can  be  identified  and  solutions  recommended.  These  solutions  shall  be  tested,  new  data  collected, 
and  system  reports  generated.  This  cycle  of  monitoring  for  reliable  outputs.  (i.en  reviewing 
reports),  documenting  changes,  and  monitoring  again  is  the  application  of  maintainability,  or 
maintenance.  Following  the  installation  of  the  prototype  AOTS,  systems  reports  shall  serve  as 
the  primary  mechanism  for  maintainability. 


3.2.5  Availability. 

The  availability  specification  in  Table  4  applies  to  the  AOTS  system. 


Table  4.  Availability  Specification 


Parameter 

Threshold 

System  wide  availability 

95% 

Computer  System  Component  availability 

96.5% 

Terminal  Component  Availability 

98% 

Printer  Components  Availability 

98% 

Digitizer  Pad  Component  Availability 

99% 

Availability  status  reports  shall  be  made  periodically  by  the  Prime  contractor. 


3.2.6  System  effectiveness  model. 

The  report  generation  capabilities  of  the  Evaluation  Subsystem  shall  be  designed  to  ensure  system 
effectiveness.  Data  resulting  from  system  functions  at  all  levels  of  the  AOTS  shall  be  collected 
and  used  in  the  generation  of status  reports.  Computer 'Supported  functions  within  the  Evaluation 
Subsystem  shall  collect,  maintain,  and  report  status  data  to  system  managers,  allowing  an  assessment 
of  the  effectiveness  of  AOTS.  Data  summaries  and  reports  shall  provide  an  analysis  of  both 
trainee  and  system  status  data  compared  to  AOTS  performance  standards  established  by  the 
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contractor  in  conjunction  with  the  Air  Force.  These  reports  can  be  used  to  identify  what  is 
working  well  and  where  deficiencies  or  problem  areas  exist.  System  reports  generated  by  the 
Evaluate  System  Component  shall  include  standard  reports  generated  at  periodic  intervals  for 
prespecified  recipients  to  address  specific  AOTS  performance  aspects  or  concerns.  Specific  data 
types  contained  in  these  reports  shall  include:  task  training  and  performance  evaluation  data; 
ancillary  and  additional  duty  training  and  performance  data;  and  career  development  course 
training  and  performance  data.  The  report  generation  capabilities  of  the  Evaluation  Subsystem 
and  system  testing  during  development  and  implementation  shall  provide  the  basis  for  the  program 
evaluation  of  AOTS. 


Formative  and  summative  testing  shall  be  based  on  the  Criteria  Acquisition  Model  (CAM),  a 
classic,  widely  used  model  from  program  evaluation  literature.  However,  due  to  the  limited 
number  of  the  prototype  AOTS  participants  a  real  classical  scientific  research  program  can  not 
be  achieved.  As  such,  a  semi-scientific  approach  with  the  CAM  shall  be  designed  to  provide 
systematic,  comprehensive  evaluation  of  a  large  scale  system  as  it  progresses  through  its  devel¬ 
opment  and  test  stages.  The  CAM  model  shall  be  used  in  developing  the  Master  Test  Plan. 

The  central  notion  of  the  model  is  that  there  are  distinct  evaluation  stages  in  the  life  of  a 
development  product  and  that  it  is  possible  to  specify  criteria  that  must  be  met  before  the  product 
can  be  advanced  to  the  next  stage  of  development.  There  are  three  facets  of  the  CAM  as  adapted 
to  AOTS: 


a.  The  formal  stages  of  the  program  (Phase  II  and  Phase  III) 

b.  The  criteria  domains  that  will  be  used  as  the  basis  for  the  evaluation  (Compliance, 
Performance,  Acceptability,  and  Suitability) 

c.  The  various  audiences  for  the  evaluation,  i.e.,  the  people  who  must  be  satisfied  by 
the  system  (AOTS  developers  and  operators,  AOTS  end  product  users,  and  Air  Force 
Management) 

These  three  facets  continually  intersect  to  provide  testing  points.  This  model  will  ensure  relevant 
testing  for  system  effectiveness  during  detailed  development  and  System  Level  Test  and  Evaluation 
(SLT&E  or  Phase  III).  The  information  resulting  from  system  reports  can  be  used  to  support 
future  AOTS  requirements  (i.e.  for  Full  Scale  Engineering  Development  -  FSED). 


3.2.7  Environmental  conditions. 

The  environmental  conditions  for  the  AOTS  system  shall  be: 

a.  Phase  II  Office  environment 

b.  Phase  III  Office  environment 

Office  environment  specifications  shall  be  within  standard  limits  of  commercially  available 
off-the-shelf  equipment.  The  limits  in  Table  3  shall  apply  as  acquisition  goals  for  the  system. 
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Table  5.  System  Component  Environment  Limits 


Component 

Op.  Temp. 
Range  (Degrees) 

Non-Op.  Temp. 
Range  (Degrees) 

Rel. 

Hum.  % 

Computer  System 

65  -  75  F 

50  -  100  F 

20  -  80 

Terminal 

55  -  90  F 

45  -  100  F 

20  -  80 

Printer 

55  -  90  F 

45  -  100  F 

20  -  80 

Digitizer  Pad 

60  -  85  F 

45  -  100  F 

20  -  80  | 

Optical  Mark  Reader 

60  -  85  F 

45  -  100  F 

20  -  80  | 

The  overall  approach  for  the  Phase  II  and  Phase  III  will  be  to  use  off-the-shelf  equipment  designed 
for  an  office  environment. 

3.2.8  Nuclear  control  requirements. 

This  paragraph  is  not  applicable  to  this  specification. 

3.2.9  Transportability. 

The  AOTS  equipment  shall  be  transportable  within  the  limits  of  commercial  off-the-shelf 
equipment.  The  following  specifics  shall  be  applied  to  each  component: 

a.  The  computer  system  shall  be  able  to  be  transported  using  air  ride  vans.  All  major 
components  of  the  computer  system  that  are  floor  standing  units  shall  have  casters 
with  the  exception  of  the  line  printer. 

b.  The  AOTS  Terminal  shall  be  capable  of  being  moved,  in  component  pieces,  by  a 
single  person.  No  single  component  shall  exceed  SO  pounds  in  weight. 

c.  The  printers  used  shall  all  be  capable  of  being  moved  and  not  exceed  SO  pounds  in 
weight. 

d.  All  optical  mark  readers  shall  be  capable  of  being  moved  and  not  exceed  SO  pounds 
in  weight. 

e.  All  digitizer  pads  shall  be  capable  of  being  moved  and  not  exceed  SO  pounds  in 
weight. 

f.  All  other  items  used  on  AOTS  shall  be  capable  of  being  moved  and  any  single  item 
shall  not  exceed  SO  pounds  in  weight 

g.  All  items  shall  be  capable  of  being  moved  using  a  dolly  type  device.  All  protruding 
items  shall  be  capable  of  being  removed  for  ease  of  moving  and  safety  during  moving. 

h.  Only  standard  type  power  plugs  shall  be  used  for  ease  of  power  relocation.  No  hard 
wiring  of  power  shall  be  permitted  (must  have  connector/plug  arrangement  for  power). 
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3.3  Design  and  construction. 

3.3.1  Materials,  processes,  and  parts. 

This  paragraph  is  not  applicable  to  this  specification. 


3.3.2  Electromagnetic  radiation. 


The  AOTS  system  off-the-shelf  components  shall  comply  with  the  FCC  regulations  Part  1 5.  subpart 
J ,  class  A  through  C  (as  applicable  to  each  component).  There  are  no  TEMPEST  requirements 
for  AOTS. 


3.3.3  Nameplates  and  product  marking. 


Nameplates  and  product  markings  of  off-the-shelf  items  shall  conform  to  standard  commercial 
practice. 


3.3.4  Workmanship. 

The  workmanship  used  on  off-the-shelf  items  shall  conform  to  standard  commercial  practice. 


3.3.5  Interchangeability. 

In  order  to  enhance  interchangeability  of  all  items  in  the  AOTS  system,  component  standardization 
is  a  requirement.  In  particular,  the  standardization  of  the  AOTS  terminal,  (including  the  basic 
unit  component,  electronic  support  cards,  monitors,  etc.),  is  a  requirement.  Use  of  equipment 
that  is  currently  in  the  United  States  Air  Force’s  Integrated  Logistics  System  (ILS)  is  strongly 
encouraged  where  applicable.  Particular  emphasis  is  placed  on  the  AOTS  terminal(s)  and  support 
equipment.  The  use  of  a  validated  Ada  compiler  is  necessary  to  ensure  highly  transportable 
software  and  interchangeability  of  software  modules  with  existing  AOTS  specific  software  and 
future  AOTS  software.  All  AOTS  specifically  developed  software  shall  use  modular  construction 
techniques.  Stand  alone  software  modules  shall  be  used  wherever  possible.  Reuse  of  software, 
where  applicable,  is  required. 


3.3.6  Safety. 

All  equipment  shall  be  safe  to  use  and  transport.  The  system  with  its  equipment  shall  be  designed 
or  chosen  to  preclude  injury  of  personnel  during  installation,  operation,  and  maintenance.  System 
safety  considerations  and  design  features  shall  be  incorporated  where  appropriate.  Any  system 
component  cannot  exceed  70  db  noise  level  without  use  of: 


a.  A  special  room  to  house  the  system  or: 

b.  Sound  absorbing  cabinet. 

Any  system  component  that  will  be  used  in  a  sensitive  office  environment  cannot  exceed  55  do 
noise  level. 


3.3.7  Human  performance/human  engineering. 

The  provisions  of  MIL-STD-1472  and  MIL-H-46855  shall  be  used  to  establish  an  effective. 
Engineering  Program.  The  Human  Engineering  Program  established  shall  be  plannc.  ,  < nt  »,i;  cc, 
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and  developed  in  conjunction  with  other  design,  development,  and  operational  functions  in  a 
manner  that  permits  the  economical  achievement  of  overall  AOTS  program  objectives.  Equipment 
design  shall  be  evaluated  to  ensure  that  optimal  man-machine  interfaces  are  accomplished,  as 
well  as  to  ensure  the  other  program  objectives  are  accomplished  for  providing: 

a.  User  Friendliness 

b.  Supportability 

3.3.7. 1  Human  Engineering  Program. 

An  effective  Human  Engineering  (HE)  Program  shall  be  established  and  maintained  throughout 
the  period  of  the  contract.  The  HE  Program  shall  be  developed  in  conjunction  with  the  Main¬ 
tainability  Program  and  Reliability  Program.  The  HE  Program  Plan  shall  provide  for  a  documented 
human  engineering  analysis  of  any  equipment  specifically  designed  for  the  AOTS.  The  human 
engineering  design  factors  of  all  existing  equipment  used  within  the  AOTS  shall  also  be  reported. 


3. 3.7.2  Human  Engineering  analysis/support. 

Human  engineering  analysis  and  support  shall  be  maintained  throughout  the  period  of  the  AOTS 
contract.  Human  engineering  analysis  support  shall  be  provided  to  designers  of  equipment 
specifically  designed  for  the  AOTS.  The  contractor  analysis  shall  be  conducted  to  aid  m  the 
selection  of  commercially  available  equipment  that  meets  AOTS  human  engineering  criteria. 
Special  emphasis  shall  be  placed  on  factors  such  as  supervisor,  trainer,  and  trainee  interface  with 
computer  and  peripheral  devices. 


3.3.7.3  Equipment,  environment,  and  facilities  design. 

It  shall  be  the  intent  to  design  the  equipment,  the  work  environment  and  the  facilities  to 
accommodate  all  personnel  between,  and  including,  the  fifth  percentile  female  and  the  ninety-fifth 
percentile  male  as  anthropometrically  defined  in  MIL-STD-1472.  Optimal  man-machine  interfaces 
shall  be  accomplished  by  gathering  data  from  such  sources  as:  expert  opinion;  questionnaires 
detailing  intended  use  of  work  space  and  equipment  therein;  industry  and  military  human  factors 
studies  and  assessments;  and  interviews  with  supervisors,  trainers  and  trainees  concerning  level 
of  understanding  of  computer  and  peripheral  device  usage.  These  data  shall  be  analyzed,  and 
results  shall  be  used  to  meet  the  unique  constraints  imposed  by  job-site  operation.  Some  of  the 
specifics  to  be  addressed,  and  accommodated  in  accordance  with  MIL-STD-1472,  are  lighting, 
noise  levels,  temperature  and  other  known  relevant  environmental  factors,  distracting  conaitions, 
work  space  requirements,  normal  viewing  distances,  equipment  placement,  use  of  equipment,  etc. 


3.3.8  Computer  programming. 

Computer  programs  and  computer  data  bases  shall  be  considered  as  software.  Software  shall  be 
categorized  as  support  or  applications  software. 

3.3.8. 1  General  requirements. 

Software  shall  meet  the  following  design,  language,  and  coding  requirements: 
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3.3.8. 1 . 1  Design  requirements. 

3.3.8. 1.1.1  Computer  program  structure. 

The  computer  program  structure  shall  consist  of  Computer  Program  Configuration  Item(s), 

Component  and  Module. 

a.  Computer  Program  Configuration  Item  (CPCI).  A  CPCI  is  the  actual  computer  program 
end  item  in  the  form  of  computer  instructions  stored  on  machine-readable  media.  A 
CPCI  shall  consist  of  one  or  more  computer  program  components. 

b.  Computer  Program  Component  (CPC).  A  CPC  is  a  functionally,  logically  distinct 
part  of  a  CPCI.  A  CPC  is  identified  for  purposes  of  convenience  in  specifying  and 
developing  a  CPCI  as  an  assembly  of  subordinate  elements.  A  CPC  consists  of  a 
logical  composition  of  one  or  more  subordinate  or  interfacing  modules. 


3.3.8.1 .1 .2  Top  Down  Design  (TDD). 

Software  shall  be  designed  in  a  top  down  manner.  The  processing  activities  of  the  system  shall 
be  identified  and  organized  beginning  with  higher  levels  of  organization,  i.e.,  top  levels.  These 
higher  levels  shall  then  be  expanded  and  broken  out  to  include  a  more  detailed  definition  of  the 
processing  activities  by  identification  of  subordinate  levels.  The  lowest  level  of  processing  shall 
correspond  to  the  module. 


3.3.8. 1 .1 .3  Top  Down  Implementation  (TDI). 

The  project  software  shall  be  implemented  in  a  top  down  manner  as  defined  herein.  Conceptually, 
top  down  implementation  proceeds  from  a  single  starting  point  while  conventional  implementation 
proceeds  from  as  many  starting  points  as  programs  in  the  design.  The  single  starting  point  does 
not  imply  that  the  implementation  must  proceed  down  the  hierarchy  in  parallel.  For  example, 
user  or  other  external  interfaces  might  be  implemented  before  some  of  the  other  partitions  to 
permit  early  demonstration  of  software  subsystem  capabilities,  partial  software  system  evaluation, 
training,  or  even  incremental  software  system  acceptance.  The  project  software  shall  be 
implemented  in  a  series  of  RELEASES  which  shall  provide  for  successive  system  capabilities. 


3.3.8.1 .2  Programming  language. 

Newly  developed  software  for  this  system  shall  be  restricted  to  the  high  order  language  Ada, 
per  MIL-STD- 1 8 1 5 A. 


3.3.8.2  Operating  System  (OS)  requirements. 

The  OS  shall  conform  to  the  following  requirements: 

a.  The  OS  shall  be  vendor-supplied,  off-the-shelf  package. 

b.  OS  augmentations  shall  be  allowed,  but  shall  be  limited  to  new  software.  No 
augmentations  shall  be  permitted  to  be  embedded  within  the  vendor  supplied  OS 
software;  a  separate  interface  shall  be  provided. 

c.  No  OS  interface  or  augmentation  software  shall  compromise  the  capability  of  tn  -* 
OS  vendor  to  provide  maintenance  over  the  life  cycle  of  the  systems. 
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d.  No  instructions  shall  be  executed  that  will  cause  the  computer  to  halt  processing 
pending  an  external  event,  except  by  the  OS.  An  exception  to  this  restriction  shall 
be  permitted  for  augmentations  to  the  OS  where  the  augmentation  is  designed  as  an 
extension  of  the  processing  control  of  the  OS.  The  exception  is  subject  to  review 
and  approval  by  the  Government. 


3.3.8.3  Firmware  requirements. 

Computer  programs  and  data  loaded  in  a  class  of  memory  that  cannot  be  dynamically  modified 
by  the  computer  during  processing  shall  be  considered  firmware.  Data  requirements  on  firmware 
shall  be  the  same  as  those  on  software.  Use  of  firmware  shall  be  subject  to  approval  by  the 
Government. 


3.3.8.4  Software  utility  services. 

This  support  software  shall  provide  the  following  minimum  capabilities: 

a.  Compilation 

b.  Assembly  which  produces  relocatable  object  code 

c.  Linking  type  loader 

d.  Generation,  maintenance,  and  initialization  of  storage  media  for  programs  and  data 

e.  Diagnostics  to  support  fault  isolation 

f.  Editing  and  debugging  tools 

All  support  software  shall  be  executable  on  the  AOTS. 


3.3.8.5  Message  generation. 

The  generation  of  error/diagnostic  messages  shall  make  a  distinction  between  (1)  the  requirements 
for  on-line  messages  to  facilitate  real-time  fault  isolation  required  to  maintain  the  system  in 
operational  status  and  (2)  the  logging  of  fault  messages  onto  system  files  for  the  category  of 
faults  which  require  isolation  and  correction,  but  which  can  be  addressed  off-line  and  do  not 
degrade  the  system  performance.  The  required  processing  time  to  identify  and  generate  an 
error/diagnostic  message  either  for  on-line  or  off-line  and  correction  shall  not  degrade  the 
operational  requirements  of  the  system. 

a.  Processor  message  and  advisory  formats  shall  not  require  additional  interpretation 
by  the  operator,  such  as  table  lookups  and  references  to  documentation,  with  the 
exception  of  lengthy  diagnostic  procedures  to  be  followed  by  the  operator  following 
an  abnormal  condition. 

b.  No  computer  program  shall  generate  a  message  or  advisory  identical  to  one  generated 
by  the  OS  or  by  another  program. 

c.  Off-line  error  messages  shall  contain  as  a  minimum  the  following  information: 

(1)  Time  error  was  detected. 

(2)  Textual  description  of  error  condition. 

(3)  Required  operator  action  where  applicable. 

(4)  Identification  of  triggering  module. 

(5)  Computer  program  or  system  execution  status  following  the  error. 
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On-line  error  messages  shall  contain  as  a  minimum  the  information  in  Items  (1),  (2),  and  (3) 
above. 


3.3  8.6  Program  coding  conventions. 

Software  shall  conform  to  required  coding  conventions  stated  below. 

a.  Each  line  of  source  code  shall  contain  no  more  than  one  statement. 

b.  Source  code  shall  be  clearly  and  conspicuously  annotated  to  explain  all  inputs,  outputs, 
branches,  and  other  items  not  implicit  in  the  code  itself. 

c.  Names  of  operator  commands,  data  entries,  program  components,  variables,  proce¬ 
dures,  and  other  software  components  shall  be  consistent  with  those  used  in  system 
design. 

d.  Code  shall  be  written  such  that  no  code  is  modified  during  execution. 


3.4  Documentation. 

3.4. 1  Development  documentation. 

The  Quality  Assurance  Program  shall  monitor  the  AOTS  design  drawings  and  specification  releases 
in  accordance  with  MIL-STD-490  and  MIL-STD-483.  The  Quality  Assurance  Program  shall  ensure 
that  configuration  records  reflect  the  outstanding  and  incorporated  changes.  Engineering  changes 
shall  be  proposed  and  submitted  as  required.  Submittals  shall  be  subject  to  prior  coordination 
with  and  surveillance  by  the  Quality  Assurance  Program. 


3.4. 1.1  Specifications. 

The  specifications  f or  AOTS  are  shown  in  Figure  10. 


3.4. 1.2  Reports. 

The  following  reports  shall  be  prepared  for  AOTS. 

System/Design  Trade  Study  Reports 
Interim  Technical  Report 
Final  Technical  Report 
Computer  Program  Test  Report(s) 

3.4. 1.3  Plans. 

The  Following  plans  shall  be  prepared  for  AOTS. 

Computer  Software  Test  Plan 
Configuration  Management  Plan 
Software  Development  Plan 
Systems  Engineering  Management  Plan 
Master  Test  Plan 

Site  Preparation  Requirements  Equipment  Installation  Plan 
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Human  Engineering  Plan 
Maintainability  Program  Plan 
Reliability  Program  Plan 
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3.4. 1.4  Other. 

The  following  other  documents  shall  be  prepared  for  AOTS. 
Computer  Programming  Standards 


3.4.2  Hardware  and  software  product  documentation. 


The  following  documentation  is  required  for  all  commercially  procured  off-the-shelf  components 
(hardware  and  sof tware)  f rom  manufacturers.  All  documentation  shall  conform  to  best  commercial 
practice.  Where  appropriate,  the  manufacturer’s  documentation  shall  be  supplemented  by  com¬ 
mercially  available  documentation  selected  on  the  basis  of  clarity,  style,  presentation  and  com¬ 
pleteness.  For  hardware  components,  the  following  documentation  is  required  (as  appropriate): 


a.  Reference  manual(s)  for  all  hardware  items  procurred  unique  to  AOTS 

b.  User  Manual(s)  for  all  hardware  items  procurred  unique  to  AOTS 

c.  Maintenance  manual(s)  for  all  hardware  items  procurred  unique  to  AOTS 

d.  Diagnostic  reference  manual(s)  for  all  hardware  items  procurred  unique  to  AOTS 

e.  Operator  manual(s)  for  all  hardware  items  procurred  unique  to  AOTS 


For  software  components,  the  following  documentation  is  required  (as  appropriate). 


a.  VAX/VMS  operating  system 

b.  VAX/ VMS  Ada  compiler 

c.  VAX/VMS  Fortran  compiler 

d.  SAS  reference  manual(s) 

e.  Alsys  Ada  compiler 

f.  Ethernet  Local  Area  Network  reference  manual(s) 

g.  Infotron  992  network  reference  manual(s) 

h.  Computer  Program  Source  Software  Listing  (as  required  by  specific  AOTS  specifically 
developed  software  and  to  be  included  in  C5  specifications). 


All  documentation  required  to  support  the  operation  and  maintenance  of  the  AOTS  program 
shall  be  maintained  during  the  life  of  the  program. 


3.5  Logistics. 

3.5.1  Maintenance. 


The  specific  responsibilities  of  personnel  and  a  detailed  description  of  maintenance  support  shall 
be  defined  in  the  Maintenance  Plan.  AOTS  Maintenance  shall  address  requirements  to  achieve 
the  following  hardware  maintenance  goals: 
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a.  Daily  check  of  mainframe  components,  including  all  elements  critical  to  support  the 
terminal  network. 


b.  Restoration  of  mainframe  equipment  availability  with  spares,  whenever  feasible. 
Repair  or  restoration  of  mainframe  equipment  shall  be  effected  within  established 
reliability  standards. 

c.  Maintenance  of  peripheral  equipment  as  recommended  by  the  hardware  vendor. 

d.  Repair  of  terminals  by  spare  interchange  until  repairs  can  be  effected.  Spares  shall 
be  maintained  at  levels  sufficient  to  achieve  a  90%  immediate  replacement  rate. 

e.  Restoration  of  AOTS  communication  line  continuity  shall  be  by  exchange  of  circuit 
when  feasible.  Repair  or  restoration  of  communications  equipment  shall  be  effected 
within  established  reliability  standards. 

3.5. 1.1  Measuring  and  testing  equipment. 

Measuring  and  testing  equipment,  used  to  ensure  that  components  conform  to  requirements  that 
are  specified  in  the  vendors  maintenance  documents  and  shall  be  calibrated  against  certified 
measurement  standards  which  have  valid  relationships  to  standards  maintained  by  the  National 
Bureau  of  Standards  (NBS).  A  calibrated  equipment  index  shall  be  maintained  for  the  equipment 
utilized.  Additionally,  equipment  users  are  responsible  for  ensuring  that  necessary  calibrations 
are  in  effect.  The  status  of  equipment  calibration  shall  be  indicated  on  the  equipment  and 
included  in  the  Quality  Assurance  audit  of  acceptance  tests.  Equipment  which  is  intended  for 
use  in  developmental,  repair,  and  maintenance  activities  where  strict  calibration  is  not  required 
shall  be  identified  as  non-calibrated.  The  Quality  Assurance  Representative  shall  determine  that 
the  necessary  certifications  exist  prior  to  returning  equipment  to  an  in-calibration  status. 


3.5.2  Supply. 

During  design  of  the  prototype  AOTS,  maximum  use  of  standard  off-the-shelf  items  shall  be 
made.  In  addition,  the  AOtS  shall  be  designed  to  be  maintained  and  repaired  using  standard 
support  equipment  to  the  maximum  extent  possible. 


3.5.3  Facilities  and  facility  equipment. 

Design  and  construction  of  real  property  facilities  will  be  accomplished  by  the  government. 


3.6  Personnel  and  training. 

3.6.1  Personnel. 


The  personnel  required  to  operate,  update,  and  maintain  the  prototype  AOTS  shall  consist  of  Air 
Force  personnel.  Air  Force  civilians,  and  contractor  personnel.  The  number  and  functions  shall 
be  determined  by  the  Air  Force  and  the  contractor  for  each  phase  of  the  AOTS  program. 


3.6.2  Training. 

Training  shall  be  conducted  at  Bergstrom  AFB,  in  Building  1S08  or  in  other  suitable  GFE  space 
The  contractor  shall  provide  training  on  the  AOTS  software  and  system  procedures  developed 
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to  meet  the  requirements  as  specified  in  this  and  lower  level  specifications.  Training  shall  occur 
during  Phase  II  and  possibly  into  Phase  III.  The  Air  Force,  assisted  by  the  contractor,  shall 
provide  training  to  the  workcenter  personnel  prior  to  and  during  Phase  III. 

3.7  Functional  area  characteristics. 

The  functional  areas  within  AOTS,  represented  by  the  five  subsystems,  were  derived  from  the 
requirements  that  AOTS  must  meet  to  improve  AF  OJT.  The  links  between  these  requirements 
and  the  functional  areas  are  depicted  in  Figure  12  and  explained  in  paragraphs  3.7.1  through 
3.7.8.  The  vertical  headers  down  the  left  side  of  Figure  12  show  AF  OJT  requirements,  and  the 
horizontal  headers  across  the  top  of  the  page  show  AOTS  functions  (subsystems).  Descriptions 
of  the  five  subsystems  are  given  in  paragraphs  3.7.9  through  3.7.13. 
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Figure  12.  Functional  Allocation  of  the  Advanced  On-the-job  Training  System  (AOTS) 


3.7. 1  OJT  administration  requirement. 

The  AOTS  is  designed  to  solve  problems  in  OJT  administration  (planning  training  requirements). 
It  changes  the  emphasis  in  OJT  from  general  career  preparation  to  position  qualification.  AOTS 
helps  the  workcenter  supervisor  plan  training  that  qualifies  trainees  for  specific  positions  while 
relieving  the  supervisor  of  much  administrative  paperwork. 

3. 7.1.1  Management  Subsystem  function. 

The  function  of  the  Management  Subsystem  is  to  assist  the  workcenter  supervisor  in  making 
accurate  selections  of  duty  position  tasks  and  in  determining  current  training  requirements. 
These  training  requirements  include: 

a.  Day-to-day  production  requirements 
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b.  Wartime/contingency  task  requirements  (e.g.,  operating  field  radio  gear,  converting 
a  bus  to  an  emergency  medical  facility  in  the  event  of  natural  disaster). 

c.  Career  Development  Course  requirements 

d.  Additional  duty  training  requirements  (e.g.,  supply  custodian,  building  custodian, 
security  monitor,  etc.) 

e.  Ancillary  training  requirements  (e.g..  Protection  of  the  President,  Standards  of 
Conduct,  etc.) 


3.7.1.2  Computer  Support  Subsystem  function. 

The  functions  of  the  Computer  Support  Subsystem  are  to: 

a.  Simplify  the  supervisor’s  job  of  determining  specific  training  requirements  for  f ull 
position  qualification  by  providing  automated  support; 

b.  Identify  and  manage  training  requirements  for  full  position  qualification,  and  hold 
constant  the  quality  of  training  across  positions; 

c.  Reduce  administrative  paperwork  involved  in  recording  training  progress. 


3.7. 1 .3  Personnel  and  Support  Subsystem  function. 

The  function  of  the  Personnel  and  Support  Subsystem  is  to  identify  task  and  training  requirements 
for  the  management  and  operation  of  AOTS. 


3.7.2  OJT  management  requirement. 

The  AOTS  is  designed  to  solve  problems  in  OJT  management.  The  supervisor  is  helped  to  maintain 
a  global  perspective  on  the  status  of  training,  trainers,  evaluators,  and  trainees. 


3.7.2. 1  Management  Subsystem  function. 

The  functions  of  the  Management  Subsystem  are  to: 


a.  Assist  with  planning  individual  and  group  OJT  programs  to  achieve  mission  objectives 
and  field  require ->«ents  without  duplication  of  effort  (i.e.  knowledge  of  available 
training  more  widuy  dispersed); 

b.  Assist  with  determining  OJT  capability  and  capacity  by  identifying  training  resources 
(people,  places,  materials);  qualified  trainers  are  identified  and  training  events  are 
scheduled; 

c.  Promote  mission  capability  and  state  of  readiness  as  a  result  of  single-point  man¬ 
agement. 


3  7.2.2  Computer  Support  Subsystem  function. 

The  function  of  the  Computer  Support  Subsystem  is  to  provide  automated  and  automatic  support 
for  the  Management  Subsystem  functions.  For  OJT  management,  it  automatically  prioritizes  t 
training  plan  based  on  the  training  capability  and  capacity  of  AOTS  at  a  given  point  in  tir 
Schedules  are  automatically  generated. 


R*t46 


SPECIFICATION  NUMBER  70S647000 
CODE  IOENT  NO.  76301 
DATE  31  Ootobw  1088 


3.7.3  Training  evaluation/assessment  requirement. 

Good  quality  training  must  be  encouraged.  With  AOTS,  this  is  promoted  because  there  are 
frequent  task  proficiency  evaluations  and  because  unit  training  effectiveness  is  assessed. 


3. 7.3. 1  Evaluation  Subsystem  function. 

The  functions  of  the  Evaluation  Subsystem  in  support  of  this  requirement  are  to  identify,  select, 
and  deliver  evaluation  instruments.  Proficiency  evaluations  are  sampled  from  all  available  tasks 
rather  than  from  a  small  group  of  the  most  commonly  performed  ones.  Evaluations  can  be 
administered  on-  or  off-line. 


3. 7.3.2  Computer  Support  Subsystem  function. 

The  functions  of  the  Computer  Support  Subsystem  are  to: 

a.  Provide  support  for  on-line  evaluation  or  printing  tests  for  off-line  use;  and 

b.  Store  test  results  and  keep  records  for  tracking  individual  as  well  as  unit  progress 
and  efficiency. 


3.7.4  System  evaluation  requirements. 

There  is  a  need  for  quality  control  over  OJT.  AOTS  enables  system  quality  assurance  and  provides 
reliability  checks. 

3.7.4. 1  Evaluation  Subsystem  function. 

The  functions  of  the  Evaluation  Subsystem  are  to: 

a.  Provide  performance  analyses  to  managers  which  show  system  effectiveness;  and 

b.  Randomly  select  individuals  for  testing  to  provide  quality  control  over  the  training 
process. 


3. 7.4.2  Computer  Support  Subsystem  function. 

The  function  of  the  Computer  Support  Subsystem  is  to  assist  the  Evaluation  Subsystem  with  the 
collection,  storage,  calculation,  and  delivery  of  reports. 


3.7.5  Data  automation  support  requirement. 

There  is  a  need  in  AF  training  to  maintain  a  single  point  of  control  over  resource  scheduling 
and  training  requirements. 

3.7.5. 1  Computer  Support  Subsystem  function. 

The  Computer  Support  Subsystem  promotes  the  linkage  of  the  subsystems  to  one  another.  This 
encourages  data  integrity  and  the  avoidance  of  data  duplication.  It  also  maintains  a  central 
point  for  reporting  of  extraction  of  OJT  information. 
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3.7.6  Specification  of  AOTS  operational  requirements. 

The  AOTS  development  effort  shall  provide  for  the  transition  of  the  prototype  to  the  AF  and 
its  expansion  to  other  AF  bases. 

3. 7.6.1  Personnel  and  Support  Subsystem  function. 

The  Personnel  and  Support  Subsystem  shall  support  four  main  functions  to  fulfill  AOTS 
operational  requirements. 

3.7.6.1.1  Personnel  requirements  function. 

The  Personnel  and  Support  Subsystem  shall  be  used  to  determine  personnel  requirements  for 
designing,  developing,  maintaining,  and  operating  the  prototype  AOTS. 


3. 7.6. 1.2  Logistics  support  requirements  function. 

The  Personnel  and  Support  Subsystem  shall  be  used  to  determine  logistics  requirements  in 
maintenance  support  (part-task  trainers,  study  carrels,  etc.)  and  computer  support  maintenance 
(computer  equipment). 


3.7.6.1 .3  Maintainability,  reliability,  and  human  factors  requirements  function. 

The  Personnel  and  Support  Subsystem  shall  be  used  to  determine  maintainability,  reliability,  and 
human  factors  requirements  for  the  AOTS. 


3.7.7  Development  of  evaluation  and  training  materials  requirement 

There  needs  to  be  a  method  for  developing  evaluation  and  training  materials  which  are  homo¬ 
geneous  across  AFSs,  but  which  can  also  reflect  local  requirements. 

3.7.7.1  Evaluation  Subsystem  function. 

The  function  of  the  Evaluation  Subsystem  is  to  support  the  development  of  behavioral  objectives, 
test  items,  and  tests  in  accordance  with  the  ISD  approach.  These  can  be  constructed  to  reflect 
global  needs  as  well  as  the  peculiarities  of  certain  locations/conditions. 


3.7.7  2  Computer  Support  Subsystem  function. 

The  function  of  the  Computer  Support  Subsystem  is  to  support  the  preparation/selection  of 
objectives,  test  items,  and  tests  in  a  user-friendly  manner. 


3. 7. 7.3  Training  Development  and  Delivery  Subsystem  function. 

The  function  of  this  subsystem  is,  in  addition  to  supporting  test  development,  to  support  the 
development  of  on-line  instructional  materials  and  their  delivery.  These  materials  can  be  c<.;. 
structed  to  reflect  global  needs  as  well  as  the  peculiarities  of  certain  locations/couditions. 
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3.7.8  Delivery  of  evaluation  and  training  materials  requirement. 

OJT  must  be  more  efficient  in  the  delivery  of  training  materials. 

3. 7.8.1  Evaluation  Subsystem  function. 

The  function  of  the  Evaluation  Subsystem  is  to  evaluate  personnel  on-line  and  off-line  for  task 
requisite  knowledge  (i.e..  knowledge  that  an  individual  must  have  in  order  to  perform  the  task) 
and  performance  skills  (i.e.,  the  requirements  for  performing  the  task). 


3. 7.8.2  Computer  Support  Subsystem  function. 

The  functions  of  the  Computer  Support  Subsystem  are  to  deliver  training  materials  and  tests, 
and  to  record  trainee  responses  and  progress,  in  order  to  improve  the  efficiency  of  OJT. 


3. 7.8.3  Training  Development  and  Delivery  Subsystem  function. 

The  function  of  this  subsystem  is  to  train  personnel  on-line  for  task  requisite  knowledge. 


3.7.9  The  Management  Subsystem. 

The  knowledge  and  performance  requirements  necessary  to  support  accomplishment  of  Air  Force 
mission  responsibilities  assigned  to  the  enlisted  force  shall  be  implemented  within  this  subsystem. 
It  is  within  this  subsystem  that  training  requirements  needed  to  qualify  enlisted  members  in 
their  assigned  jobs  shall  be  defined  and  managed.  The  Management  subsystem  shall  be  comprised 
of  three  major  components:  1)  the  Training  Requirements  Management  Component,  2)  the  Airman 
Training  Management  Component,  and  3)  Maintain  Resource  Inventories  and  Schedule  Events 
Component.  Tkese  components  shall  comprise  three  computer-based  management  methodologies: 
1)  a  methodology  for  identifying  Air  Force  Speciality  (AFS)  and  other  training  required  to 
achieve  position  qualification  within  an  assigned  operation  duty  position;  2)  a  methodology  for 
managing  and  recording  airman  training  progress  toward  task  proficiency  and  position  quali¬ 
fication  within  an  operation  duty  position,  and  3)  a  methodology  for  identifying  performance, 
training,  and  evaluation  resources  and  scheduling  training.  A  detailed  description  of  this 
subsystem  is  found  in  Specification  70S647100. 


3.7.10  The  Evaluation  Subsystem. 

The  Evaluation  Subsystem  shall  provide  a  computer-based  development  and  implementation 
process,  within  which  evaluation  materials  and  procedures  shall  be  developed.  The  Evaluation 
Subsystem  shall  assess  both  task  proficiency  and  trainee  knowledge.  It  shall  also  collect  and 
analyze  data  that  is  related  to  the  efficiency  and  cost-effectiveness  of  job-site  training.  This 
subsystem  shall  present,  store,  distribute,  and  limit  access  to  computer-based  evaluation  data  and 
generate  all  system  evaluation  reports.  The  subsystem  shall  be  composed  of  four  components:  1): 
the  Manage  Evaluation  Instrumentation  Component;  2)  the  Evaluate  Performance  Component;  3) 
the  Apply  Training  Quality  Control  Component  and  4)  the  Evaluate  System  Component.  A 
detailed  description  of  this  subsystem  is  found  in  Specification  70S647300. 


3.7.11  The  Computer  Support  Subsystem. 

This  subsystem  shall  be  the  computer  hardware  and  software  required  to  support  the  management, 
development,  delivery,  and  evaluation  of  job-site  training  and  performance  evaluation  within 
Air  Force  OJT.  The  other  four  subsystems  within  AOTS  shall  provide  requirements/needs  for 
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hardware,  software,  and  database  needs.  The  Computer  Support  Subsystem,  in  turn,  shall  provide 
the  computer  hardware  equipment,  automated  software  services,  and  the  repository  space  for 
data  necessary  to  satisfy  the  AOTS  requirements  of  the  other  subsystems.  This  subsystem  shall 
be  composed  of  two  components:  1)  the  Hardware  Component,  and  2)  the  Software  Component 
A  detailed  description  of  this  subsystem  is  found  in  Specification  70S647400. 


3.7.12  The  Personnel  and  Support  Subsystem. 

This  subsystem  shall  identify  and  designate  the  personnel  required  to  perform  the  jobs  created 
or  changed  by  the  other  AOTS  subsystems.  It  shall  identify  maintenance  support  requirements 
for  training  devices  and  associated  computer  hardware.  The  subsystem  shall  be  composed  for 
two  components:  1)  the  Logistic  Support  Requirements  Component;  and  2)  the  Maintainability, 
Reliability,  and  Human  Factors  Requirements  Component.  A  detailed  description  of  this  sub¬ 
system  is  found  in  Specification  70S647S00. 


3.7. 1 3  The  Training  Development  and  Delivery  Subsystem. 

This  subsystem  shall  provide  a  computer-based,  interactive,  instructional  materials  development 
process  for  writing,  evaluating,  and  revising  appropriate  training  materials.  It  shall  also  provide 
for  computer  presentation,  storage,  distribution,  and  access  control  of  training  materials.  The 
training  materials  development  environment  shall  be  supportive  of  requirements  imposed  by  AFM 
50-2  and  AFR  30-17,  unless  deviations  from  such  regulations  and  manuals  have  been  approved 
by  the  AF.  Training  materials  developed  shall  be  stored  and  delivered  in  support  of  trainee 
position  qualification.  Access  control  of  training  materials  shall  comply  with  requirements  set 
forth  in  AFR  205-1.  Individual  trainee  assignments  shall  be  generated  which  are  predicated  on 
current  trainee  position  qualification  status.  Data  reflecting  individual  training  status  shall  be 
generated  as  training  and  evaluation  events  occur.  The  subsystem  shall  be  composed  of  two 
components:  1)  the  Training  Development  Component,  and  2)  the  Training  Delivery  Component. 
A  detailed  description  of  this  subsystem  is  found  in  Specifications  70S647200,  70S647201,  and 
70S647202.  This  subsystem  will  make  maximum  use  of  the  ISS  software  that  will  be  provided 
by  the  Government. 


3.8  Precedence. 

This  specification  shall  take  precedence  over  any  document  referenced  herein.  The  contracting 
activity  shall  be  notified  immediately  of  each  instance  of  conflicting,  or  apparently  conflicting, 
requirements. 


4  QUALITY  ASSURANCE  PROVISIONS 
4.1  General. 


The  quality  assurance  provisions  of  this  section  shall  be  directed  towards  verifying  compliance 
with  the  system  requirements  stated  in  this  specification.  All  verifications/acceptance  activities 
shall  be  accomplished  as  prescribed  in  this  section,  unless  expressly  waived  by  the  government. 
There  will  be  two  types  of  testing  of  AOTS.  The  first  type  ensures  that  the  system  hardware 
and  software  perform  as  specified  in  Section  3  of  the  specification.  That  is,  the  system  should 
operate  as  specified  and  should  meet  all  performance  requirements.  The  second  type  of  testinc 
is  the  proof -of -concept  for  the  system.  The  concept  must  be  tested  that  AOTS  meets  the  OJT 
requirements  for  the  AF.  This  is  done  by  continual  monitoring  of  the  prototype  AOTS  as  it  is 
incrementally  developed  and  activated  at  the  test  sites  (Bergstrom  AFB  and  Ellington  ANG3V 
By  administering  questionnaires  and  interviews  to  workcenter  managers,  instruct’onai  dcsig 
personnel,  and  other  relevant  subjects,  as  well  as  through  other  testing  methods,  the  detcMed 
design  of  AOTS  can  be  modified  to  more  closely  fit  AF  needs  and  enhance  the  ability  of  AO  IS 
to  meet  OJT  needs  AF  wide.  Tests  shall  be  conducted  at  Bergstrom  AFB  and  at  Lllington  /iNGB, 
both  in  Texas.  There  are  four  critical  issues  in  regard  to  testing  AOTS.  These  issue ,  ri>  «• 
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the  proper  functioning  of  system  hardware  and  software  and  arc  exclusive  of  hardware  and 
software  testing  alone.  They  relate  to  a  system's  overall  capability  and  must  be  addressed  in  an 
evaluation  of  the  system,  as  a  whole.  Furthermore,  they  are  of  primary  importance  to  the  decision 
authority  for  AOTS. 


a.  Critical  Issue  No.  1:  Compliance  -  whether  the  prototype  AOTS  meets  the  system 
specification  requirements. 


Supporting  Rationale  -  The  critical  question  associated  with  compliance  is: 

(1).  Does  the  prototype  AOTS  meet  the  requirements  of  the  system 
specification,  while  operating  in  an  operational  environment?  This 
is  followed  by  the  following  subquestions: 

(a).  Does  AOTS  provide  the  required  interfaces  with  external 
entities  (system  specification  sections  3.1.5.1.1-2,  3.1.5.1.4-6,  and 
3.1.5.1.8-10)? 


(b).  Does  AOTS  meet  the  response  performance  goals  (system 
specification  section  3.2. 1.1)? 


(c).  Does  AOTS  meet  the  hardware  reliability  goals  (system  speci¬ 
fication  section  3.2.3. 1)? 


(d).  Does  AOTS  meet  the  hardware  maintainability  goals  (system 
specification  section  3.2.4.2)? 


(e).  Does  AOTS  provide  required  system  availability  (system 
specification  section  3.2.5)? 


b.  Critical  Issue  No.  2:  Performance  -  whether  the  prototype  AOTS  is  more  effective 
and  efficient  than  conventional  OJT. 


Supporting  Rationale  -  The  critical  questions  associated  with  performance  are: 


(1).  Do  trainees  achieve  position  qualification  at  a  faster  rate  under  the 
prototype  AOTS?  This  is  followed  by  the  following  subquestions: 

(a).  Do  trainees  become  position  qualified  sooner  under  AOTS  than 
under  conventional  OJT? 


(b).  Does  the  AOTS  improve  the  rate  at  which  airmen  progress 
toward  position  qualification? 


(2).  Do  those  trainees  who  are  trained  under  the  prototype  AOTS  perform 
better  in  duty  position  tasks?  This  is  followed  by  the  following 
subquestion: 


(a).  Does  the  AOTS  improve  the  performance  of  airmen  on  quality 
control  and  other  standard  evaluations? 


(3). 


Do  the  workcenters  usin^  the  prototype  AOTS  contribute  more  to 
mission  readiness?  This  is  followed  by  the  following  subquestion: 
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(a).  Do  the  repeat  maintenance  actions  within  the  aircraft  main¬ 
tenance  workcenters  decrease  after  the  AOTS  is  implemented? 


(4) .  Do  the  workcenters  using  the  prototype  AOTS  contribute  more  to 

mission  readiness?  This  is  followed  by  the  following  subquestions: 

(a) .  Do  base  crime  statistics  attributed  to  the  applicable  security 

police  workcenters  improve  after  the  AOTS  is  implemented? 

(b) .  Do  supervisors  feel  that  they  contribute  more  to  mission 

readiness,  as  a  result  of  AOTS  implementation? 

(5) .  Does  the  prototype  AOTS  require  less  time  for  management  of  AFS 

related  training?  This  is  followed  by  the  following  subquestion: 

(a).  How  much  time  is  spent  managing  AFS  related  training  under 
the  AOTS  vs.  conventional  OJT? 


(6).  Do  users  feel  that  training  under  the  prototype  AOTS  is  more 
effective?  This  is  followed  by  the  following  subquestion: 

(a).  Do  users  feel  that  trainees  who  have  been  trained  under  the 
AOTS  are  more  capable  of  performing  AFS  task  requirements 
than  trainees  trained  under  conventional  OJT? 


Critical  Issue  No.  3:  Suitability  -  whether  the  prototype  AOTS  solves  the  problems 
experienced  with  conventional  OJT  and  is  suited  for  deployment  throughout  the  Air 
Force. 


Supporting  Rationale  -  The  critical  questions  associated  with  suitability  are: 


(1) .  Does  the  prototype  AOTS  result  in  more  time  spent  on  AFS  task 

training,  for  those  airmen  who  are  not  yet  fully  position  qualified? 
This  is  followed  by  the  following  subquestion: 

(a).  Does  the  time  spent  on  task  training  increase  under  AOTS? 

(2) .  Are  supervisors  better  able  to  determine  training  and  evaluation 

requirements  under  the  prototype  AOTS  than  under  conventional 
OJT?  This  is  followed  by  the  following  subquestions: 

(a).  Do  supervisors  feel  they  are  better  able  to  determine  training 
and  evaluation  requirements  under  AOTS  than  under  conven¬ 
tional  OJT? 


(b).  Do  the  AOTS  documents  enable  supervisors  to  do  a  better  job 
of  determining  training  and  evaluation  requirements? 

(3).  Are  supervisors  better  able  to  determine  trainee  qualifications  under 
the  prototype  AOTS  than  under  conventional  OJT?  This  is  followed 
by  the  following  subquestions: 
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(a).  Do  supervisors  feel  the  prototype  AOTS  helps  them  to  determine 
trainee  qualifications? 


(b).  Do  supervisors  use  the  AOTS  products  in  determining  trainee 
qualifications? 

Does  the  prototype  AOTS  provide  standardized  methods  for  evalu¬ 
ating  task  performance  in  an  operational  setting?  This  is  followed 
by  the  following  subquestions: 

(a).  Do  the  AOTS  evaluation  methods  meet  the  needs  of  the 
workcenters? 


(b).  Does  the  AOTS  provide  standardized  evaluation  products  and 
processes  for  the  operational  setting  of  OJT? 


Does  the  prototype  AOTS  compensate  for  the  limited  availability  of 
operational  equipment  for  training?  This  is  followed  by  the  following 
subquestions: 

(a).  Is  the  availability  of  operational  equipment  a  problem  for 
training? 


(b).  Does  the  prototype  AOTS  provide  effective  methods  to  com¬ 
pensate  for  the  limited  availability  of  operational  equipment 
for  training? 

Does  the  prototype  AOTS  provide  improved  capabilities  for  managing 
the  availability  of  qualified  trainers?  This  is  followed  by  the 
following  subquestion: 


(a).  Do  users  feel  that  the  prototype  AOTS  provides  improved 
capabilities  for  managing  the  availability  of  qualified  trainers, 
as  compared  with  conventional  OJT? 


Does  the  prototype  AOTS  enhance  trainer  effectiveness?  This  is 
followed  by  the  following  subquestions: 


(a).  Do  users  feel  that  the  prototype  AOTS  enhances  trainer 
effectiveness? 


(b).  Do  the  AOTS  products  and  processes  make  trainers  more 
effective? 


Does  the  prototype  AOTS  reduce  the  administrative  burden  associated 
with  the  management  of  training?  This  is  followed  by  the  following 
subquestions: 


(a).  Do  users  feel  that  the  prototype  AOTS  reduces  the  adminis¬ 
trative  burden  associated  with  the  management  of  training? 


(b).  Do  users  spend  less  time  on  administrative  jobs  related  to 
management  of  training  under  AOTS  than  under  conventional 
OJT? 
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(8).  Does  the  prototype  AOTS  provide  improved  capabilities  for  imple¬ 
menting  state-of-the-art  instructional  technology  into  the  operational 
environment?  This  is  followed  by  the  following  subquestions: 

(a) .  Do  users  feel  that  the  prototype  AOTS  provides  improved 

capabilities  for  implementing  instructional  technology  into  the 
operational  environment? 

(b) .  Does  AOTS  provide  state-of-the-art  instructional  technology 

for  OJT? 


(9).  Can  the  prototype  AOTS  be  deployed  to  the  operational  workcenters? 
This  is  followed  by  the  following  subquestion: 

(a).  Do  users  of  the  prototype  AOTS  feel  the  prototype  AOTS  can 
be  deployed  to  the  operational  workcenters? 

Critical  Issue  No.  4:  Acceptance  -  whether  the  prototype  AOTS  is  user  friendly  and 
liked  by  the  users. 

Supporting  Rationale  •  The  critical  questions  associated  with  acceptance  is: 

(1).  Do  users  of  the  prototype  AOTS  like  the  capabilities  that  are  provided 
for  managing  training?  This  is  followed  by  the  following  sub¬ 
questions: 

(a).  Are  the  capabilities  provided  adequate  for  managing  training 
within  the  AOTS  workcenters? 


(b).  Do  the  users  like  the  management  products  generated  by  the 
AOTS? 


(c).  How  satisfied  are  users  with  the  training  they  received  on 
operating  the  AOTS  management  programs  and  maintaining 
management  data? 

(2) .  Do  users  of  the  prototype  AOTS  like  the  capabilities  that  are  provided 

for  delivering  training?  This  is  followed  by  the  following  sub¬ 
questions: 

(a) .  How  satisfied  are  users  with  the  capabilities  provided  for 

authoring  training  materials? 

(b) .  How  satisfied  are  users  with  the  functions  of  the  training 

delivery  programs? 

(c) .  How  satisfied  are  users  with  the  training  they  received  on 

operating  the  training  delivery  programs? 

(3) .  Do  users  of  the  prototype  AOTS  like  the  capabilities  tint  are  provider1 

for  evaluating  training?  This  is  followed  by  the  following  su’ 
questions: 
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(a) .  How  satisfied  are  users  with  the  capabilities  provided  for 

evaluating  training? 

(b) .  How  satisfied  are  users  with  the  functions  of  the  evaluation 

programs? 

(c) .  How  satisfied  are  users  with  the  training  they  received  on 

operating  the  AOTS  evaluation  programs? 

4.1.1  Responsibility  for  tests. 

The  contractor  shall  be  responsible  for  conducting  all  tests.  Both  contractor  and  AF  personnel 
shall  conduct  and  participate  in  tests.  A  determination  shall  be  made,  through  negotiations 
between  the  AF  and  the  contractor,  which  tests  will  require  AF  participation  and  the  level  of 
participation  required. 


4.1.2  Special  tests  and  examinations. 

This  section  is  not  applicable  to  this  specification. 


4.2  Quality  conformance  inspections. 

Verification  of  the  requirements  of  section  3  of  this  specification  shall  include  the  use  of 
inspection,  analysis,  test,  and  demonstration  (or  evaluation).  Verification  shall  be  accomplished 
in  accordance  with  the  Software  Test  Plan  and  the  Master  Test  Plan. 

4.2.1  Test  matrix. 

The  test  matrix.  Table  6,  establishes  the  inspection,  analysis,  test,  and  demonstration  (evaluation) 
requirements  for  each  of  the  requirements  specified  in  section  3.  All  verifications  shall  be 
conducted  to  satisfy  as  many  matrix  entries  as  possible. 
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Section 


Table  6.  Quality  Conformance  Method* 


N/A  Inspection  Analysis 


3. 1.5. 1. 2 

3. 1.5.1. 3 

3. 1.5. 1.4 


3. 1.5.1. 5 

3. 1.5. 1.6 

3. 1.5. 1.7 


3. 1.5. 1.8 

3.1. 5.1.9 

3.1.5.1.10 


3.1.5.1.11 

3.1.5.1.12 

3. 1.5.1. 13 


3.1.5.1.14 

3.1.5.1.15 

3.1.5.1.16 


3.1.5.1.17 
3. 1.5.2 
3. 1.5. 2.1 


3. 1.5. 2.2 

3.1. 5.2.3 

3.1. 5. 2.4 


3.1. 5.2.5 

3.1. 5.2.6 

3.1. 5.2.7 


3.1. 5.2.8 

3.1. 5.2.9 

3.1.5.2.10 


3.1. 5.2.1 1 

3.1.5.2.12 

3.1.5.2.13 


3.1.5.2.14 

3.1.5.2.15 

3.1.5.2.16 
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Table  6.  Quality  Conformance  Methods,  cont. 


Section 


3.1.5.2.17 

3.1.5.2.18 

3.1.5.2.19 


3.1.5.2.20 
3.1.6 
3. 1.6.1 


3.1  6.2 

3. 1.6.2. 1 

3. 1.6.2. 1.1 


3. 1.6.2. 1.2 

3.1. 6.2.2 

3.1. 6.2.3 


3. 1.6.7  3.1 

3.1. 6.2.3. 1. 1 

3.1. 6.2.3. 1.2 


3.1. 6.2.3. 1.3 

3.1. 6.2.3. 1.4 

3.1.6.2.3.1.5 


3.1.6.2.3.2 
3.1.7 
3. 1.7.1 


3. 1.7.2 

3. 1.7.3 
3.2 


3.2.1 

3.2. 1.1 

3.2. 1.2 


3.2.1. 2.1 

3.2.1.2.2 

3.2. 1.2.3 
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Table  6.  Quality  Conformance  Methods,  cont. 
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4.2.2  Verification  definitions. 


a.  Inspection  is  defined  as  the  examination  by  visual  or  other  equivalent  means  to 
determine  conformity  to  specified  requirements  or  performance  characteristics.  The 
criterion  of  acceptance  for  each  inspection  item  will  be  expert  consensus  or  equivalent, 
as  agreed  upon  by  the  contractor  and  Air  Force  PM. 


b.  Analysis  is  defined  as  the  determination  of  conformity  to  specified  requirements  or 
performance  characteristics  by  use  of  descriptive  data  gathering,  analysis,  and 
interpretation  techniques.  The  criterion  of  acceptance  for  each  analysis  item  will 
be  a  quantifiable  measure  or  composite  measure  as  agreed  upon  by  the  contractor 
and  Air  Force  PM 


c.  Test  is  defined  as  the  determination  of  conformity  to  specified  requirements  or 
performance  characteristics  under  controlled  conditions  with  specified  inputs,  out¬ 
puts,  and  data  analysis  methods  or  procedures.  The  criterion  of  acceptance  for  each 
test  item  will  be  a  quantifiable  measure  or  composite  measure,  to  a  specified  statistical 
probability  level,  as  agreed  upon  by  the  contractor  and  the  Air  Force  PM 

d.  Demonstration  is  defined  as  determination  of  conformity  to  specified  requirements 
or  performance  characteristics  under  controlled  or  uncontrolled  conditions,  with 
specified  inputs,  outputs,  and  data  analysis  methods  or  procedures.  Additionally, 
demonstration  may  include  the  variation  of  inputs  and  the  determination  of  resultant 
dif  f erential  outputs  and  consequences,  and  may  include  either  descriptive  or  statistical 
treatments  of  individual  or  composite  data  measures.  The  criterion  for  acceptance 
of  each  evaluation  item  will  be  a  quantifiable  measure,  composite  measure,  index, 
or  profile  configuration  as  agreed  upon  by  the  contractor  and  Air  Force  PM. 


5  PREPARATION  FOR  DELIVERY 


5.1  General  requirements. 

The  equipment  shall  be  packaged  for  protection  against  corrosion,  ether  forms  of  deterioration, 
and  physical  damage  during  shipment  from  the  manufacturer’s  plant  to  the  point  of  destination. 
The  protection  processes  to  be  used  shall  be  the  minimum  required  for  adequate  protection  under 
the  conditions  of  normal  handling,  shipment  by  common  carrier  and  anticipated  storage.  Best 
commercial  standards  shall  apply.  All  equipment/materials  susceptible  to  damage  from  EMR, 
temperature  variations,  humidity  extremes,  static  discharge  and  shock  shall  be  appropriately 
shielded  and  documented  prior  to  packaging  and  shipment.  Magnetic  media  such  as  floppy  disks, 
magnetic  tapes  or  disk  packs  shall  be  isolated  and  shielded  as  far  as  practical  from  those  factors 
which  may  damage  or  alter  the  integrity  of  the  data. 


5.2  Detailed  preparation. 

5.2. 1  Preservation  and  packaging. 

Measuring  and  testing  equipment,  used  to  ensure  that  components  conform  to  requirements  th 
are  specified  in  the  vendors  maintenance  documents  and  shall  be  calibrated  against  ccrtifi. 
measurement  standards  which  have  valid  relationships  to  standards  maintained  by  the  Niiiorut 
Bureau  of  Standards  (NBS).  A  calibrated  equipment  index  shall  be  maintained  for  the  equipmer 
utilized.  Additionally,  equipment  users  are  responsible  for  ensuring  that  necess.try  ak'>.  t-o  . 
are  in  effect.  The  status  of  equipment  calibration  shall  be  indicated  on  thv  cq„  .  an 
included  in  the  Quality  Assurance  audit  of  acceptance  tests.  Equipment  whk‘  k-  .  c  ,~  l\ 
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use  in  developmental,  repair,  and  maintenance  activities  where  strict  calibration  is  not  required 
shall  be  identified  as  non-calibrated.  The  Quality  Assurance  Representative  shall  determine  that 
the  necessary  certifications  exist  prior  to  returning  equipment  to  an  in-calibration  status. 

All  materials  shall  be  packaged  in  containers  constructed  to  provide  a  compact  and  non-shifting 
load.  Such  containers  may  be  of  subcontractor's  supply  or  manufacturer’s  crates/cartons.  Adequate 
cushioning,  blocking,  bracing,  or  strapping  shall  be  provided  so  as  to  prevent  damage  due  to 
shock,  vibration,  or  impact  during  shipment,  handling,  and  storage. 


5.2.2  Packing. 

Units  packaged  in  accordance  with  5.2.1  shall  be  packed  in  containers  which  will  ensure  acceptance 
by  common  carrier  and  safe  delivery  at  destination.  Shipping  containers  shall  comply  with 
common  carrier  rules  and  regulations  as  applicable  to  the  mode  of  transportation,  and  may  be 
the  supplier’s  commercial  practice  when  such  meets  these  requirements.  Packing  shall  be  in 
accordance  with  best  commercial  practices  or  special  instructions  based  on  unique  requirements 
for  an  item. 


5.2.3  Marking  for  shipment. 

Interior  packages  and  exterior  shipping  containers  shall  be  marked  in  accordance  with  the  best 
commercial  practices  or  special  instructions. 

6  NOTES 

This  section  is  not  applicable  to  this  specification. 

7  N/A 

8  N/A 

9  N/A 
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10 

N/A 

11 

N/A 

12 

N/A 

13 

N/A 

14 

N/A 

15 

N/A 

16 

N/A 

17 

N/A 

18 

N/A 

19  DATA  FLOW  DIAGRAM 


Data  Flow  Diagram  Symbols:  Description  of  the  symbols  appearing  in  the  following  data  flow 
diagram: 

a.  Square:  an  external  entity  symbol.  A  source  and/or  destination  of  data  outside  the 
system,  subsystem,  component,  or  process  being  described.  Numbered  references  of 
the  external  entity  are  contained  m  the  top  left  corner  of  the  square. 

b.  Arrow:  data  flow  symbol.  A  pathway  along  which  data  moves  into,  around,  and  out 
of  the  system  or  component.  Each  data  flow  is  labeled.  Definitions  of  data  flows 
are  organized  alphabetically  in  Section  20. 


c. 


Rounded  rectangle:  a  process  symbol.  A  function  of  the  system  or  component  which 
logically  "transforms"  data.  The  processes  are  numbered  numerically,  beginning  with 

"1",  on  each  data  flow  diagram.  The  numbering -  '  “ -  * - *  ’  ■’ —  ’ 

scheme  to  trace  from  one  process  to  another. 


system  follows  a  logical  decimal 
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Figure  14.  System  Data  Flow  Diagram 
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20  DATA  DICTIONARIES 

20.  f  Scope. 

This  appendix  contains  data  defining  the  processes,  data  flows,  and  external  entities  pertinent 
to  the  AOTS,  as  follows: 

a.  External  entities:  The  external  entities  are  identified.  External  entities  are  organized 
numerically. 

b.  Data  processes:  The  highest  level  processes  were  described  in  the  text  of  Section  3.7. 
Those  descriptions  are  repeated  here.  They  are  organized  numerically. 

c.  Data  flows:  Defines  the  content  of  the  data  flows  depicted  on  each  DFD.  They  are 
organized  alphabetically.  Each  data  flow  entry  explains  the  data  source,  data 
destination,  and  structure  (when  known). 


External  Entities 

1  Air  Force  Occupational  Me? surement  Center  (AFOMC) 

2  Publication  Management  System 

3  Extension  Course  Institute  (ECI) 

4  Air  Force  Military  Personnel  Center  (AFMPC) 

5  Operational  Units 

6  Air  Force  Human  Resources  Laboratory  (AFHRL) 

7  Personnel  Data  System  (PDS) 

8  Off-Line  Training  Activities 
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PROCESS: 

LEVEL  OF  AUTOMATION: 
DESCRIPTION: 

DATA  SOURCE: 

DATA  DESTINATION: 
STRUCTURE: 


PROCESS: 

LEVEL  OF  AUTOMATION: 
DESCRIPTION: 


DATA  SOURCE: 
DATA  DESTINATION: 
STRUCTURE: 
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1.0,  Manage  Training 
N/A 


The  knowledge  and  performance  requirements  to  support 
accomplishment  of  Air  Force  mission  responsibilities  assigned  to 
the  enlisted  force  shall  be  identified  within  this  subsystem. 
Training  requirements  for  qualifying  enlisted  members  in  their 
assigned  jobs  shall  be  defined  ana  managed  in  this  subsystem. 

N/A 

N/A 


This  subsystem  includes  processes: 

-  Process  1.1,  Manage  Training  Requirements 
•  Process  1.2,  Manage  Airman  Training 

-  Process  1.3,  Manage  Resources  and  Training 

Schedules 


2.0,  Evaluate  Trainees  and  Training  System 
N/A 

The  Evaluation  Subsystem  shall  provide  computer-based  devel¬ 
opment  and  delivery  processes,  within  which  evaluation  materials 
and  procedures  shall  be  developed.  The  Evaluation  Subsystem 
shall  assess  both  the  task  proficiency  and  the  task  requisite 
knowledge  of  the  trainee.  This  subsystem  shall  present,  store, 
distribute,  and  limit  access  to  computer-based  evaluation  instru¬ 
ments  and  items.  It  shall  collect  task  performance  evaluation  data 
and  generate  all  system  evaluation  reports. 

N/A 

N/A 


This  subsystem  includes  processes: 

-  Process  2.1,  Manage  Evaluation  Instrumentation 

-  Process  2.2,  Evaluate  Performance 

-  Process  2.3,  Apply  Training  Quality  Control 

-  Process  2.4,  Evaluate  System 
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PROCESS: 

LEVEL  OF  AUTOMATION: 
DESCRIPTION: 


DATA  SOURCE: 
DATA  DESTINATION: 
STRUCTURE: 


PROCESS: 

LEVEL  OF  AUTOMATION: 
DESCRIPTION: 

DATA  SOURCE: 

DATA  DESTINATION: 
STRUCTURE: 


3.0,  Provide  Computer  Support 
N/A 

This  subsystem  shall  be  the  computer  hardware  and  software 
required  to  support  the  management,  development,  delivery,  and 
evaluation  of  job-site  training  and  performance  evaluation  within 
Air  Force  OJT.  The  other  four  subsystems  within  AOTS  shall 
provide  functional  hardware,  software,  and  database  requirements 
to  the  Computer  Support  Subsystem.  The  Computer  Support 
Subsystem,  in  turn,  snail  provide  the  computer  hardware  equip¬ 
ment,  automated  software  services,  and  the  repository  space  for 
data  necessary  to  satisfy  the  AOTS  requirements  of  the  other 
subsystems. 

N/A 

N/A 


This  subsystem  includes  processes: 

-  Process  3.1,  Manage  Hardware 

-  Process  3.2,  Manage  Software 


4.0,  Manage  Personnel  and  System  Support 
N/A 

This  subsystem  shall  consist  of  the  processes  and  procedures 
required  for  providing  the  personnel,  maintenance,  and  logistic 
support  required  for  the  operation  of  the  prototype  AOTS. 

N/A 

N/A 

This  subsystem  includes  processes: 

-  Process  4.1,  Identify  Logistics  Support 

Requirements 

-  Process  4.2,  Identify  Maintainability, 

Reliability,  and  Human  Factors  Requirements 
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PROCESS: 

LEVEL  OF  AUTOMATION: 
DESCRIPTION: 


DATA  SOURCE: 
DATA  DESTINATION: 
STRUCTURE: 


3.0,  Manage  Training  Development  and  Delivery 
N/A 

This  subsystem  shall  provide  a  computer-based,  interactive 
instructional  materials  development  process  within  which 
appropriate  training  materials  can  be  written,  evaluated,  and 
revised.  It  shall  also  provide  for  computer  presentation,  storage, 
distribution,  and  access  control  of  training  materials  and  trainee 
data.  The  training  materials  development  environment  shall  be 
supportive  of  requirements  imposed  by  AFM  30-2  and  AFP  50-58, 
unless  deviations  from  such  regulations  and  manuals  have  been 
approved  by  the  AF.  Training  materials  developed  shall  be  stored 
and  delivered  in  support  of  trainee  position  qualification.  Access 
control  of  training  shall  comply  with  requirements  set  forth  in 
AFR  205-1. 

N/A 

N/A 


This  subsystem  includes  processes: 

-  Process  5.1,  Manage  Training  Development 

-  Process  5.2,  Manage  Training  Delivery 
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DATA  FLOW: 
DESCRIPTION: 


DATA  SOURCE: 
DATA  DESTINATION: 
STRUCTURE: 


DATA  FLOW: 
DESCRIPTION: 


DATA  SOURCE: 
DATA  DESTINATION: 
STRUCTURE: 


AFS  Task  Analysis 


Occupational  Survey  (OS)  data  and  Comprehensive  Occupational 
Data  Analysis  Program  (CODAP)  task  data  are  passed  from 
USAFOMC  to  Process  1.0,  Manage  Trainings,  and  analyzed.  Tasks 
required  to  be  performed  in  a  duty  position,  but  which  are  not 
found  on  the  OS  or  CODAP  data,  are  analyzed,  added  to  the 
Master  Task  List  (MTL)  of  the  AOTS,  and  passed  back  to 
USAFOMC. 


External  Entity  1,  USAFOMC;  Process  1.0,  Manage  Training 
Process  1.0,  Manage  Training;  External  Entity  1,  USAFOMC 
Structure  will  be  determined  in  the  B-l,  B-5,  and  C-5  specifications. 


Air  Force  Publications/Changes 


Data  identifying  Air  Force  Publications,  such  as  regulations, 
technical  orders,  manuals,  and  other  documents,  that  provide 
performance  and  proficiency  requirements  for  task  performance. 
Also  included  are  notices  of  publication  changes  that  may  affect 
performance  and  proficiency  requirements.  Publications  are 
identified  by  identification  code,  title,  volume  number  (if 
applicable),  and  breakdowns  in  subunits  of  the  publication  (as 
appropriate  to  the  particular  task). 

External  Entity  2,  Publication  Management  System 

Process  1.0,  Manage  Training 

Structure  will  be  determined  in  the  B-l,  B-5,  and  C-5  specifications. 
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DATA  FLOW:  AOTS  Training  Requirements 

DESCRIPTION:  Training  identified  within  the  Personnel  and  Support  Subsystem 

processes  as  being  required  for  the  functioning  of  the  AOTS  are 
passed  on  to  the  Training  Management  Subsystem  for  imple¬ 
mentation. 

DATA  SOURCE:  Process  4.0,  Manage  Personnel  and  System  Support 

OATA  DESTINATION:  Process  1.0,  Manage  Training 

STRUCTURE:  Structure  will  be  determined  in  the  B-l,  B-5,  and  C- 5 

specifications. 


DATA  FLOW:  CDC  Requirements 

DESCRIPTION:  Requirements  for  airmen  to  enroll  in  Career  Development  Courses 

(CDC)  are  passed  from  the  external  entity.  Extension  Course 
Institute  (ECI),  to  the  Training  Management  Component.  The 
courses  provide  general  knowledge  for  particular  Air  Force 
Specialties  (AFSs).  Requirements  by  AFS  skill  level  are  listed  in 
the  ECI  catalog. 

DATA  SOURCE:  External  Entity  3,  Extension  Course  Institute 

DATA  DESTINATION:  Process  1.0,  Manage  Training 

Structure  will  be  determined  in  the  B-l,  B-5,  and  C-5  specifications. 


STRUCTURE: 
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DATA  FLOW:  Computer  Support 

DESCRIPTION:  This  data  flow  consists  of  the  software  components  and  data 

required  for  the  functioning  of  the  other  AOTS  subsystems. 

DATA  SOURCE:  Process  3.0,  Provide  Computer  Support 

DATA  DESTINATION:  Process  1.0,  Manage  Training;  Process  2.0,  Evaluate  Trainees  and 

Training  System;  Process  5.0,  Manage  Training  Development  and 
Delivery 

STRUCTURE:  Structure  will  be  determined  in  the  B-l,  B-5,  and  C-5  specifications. 


DATA  FLOW:  Evaluation  Requirements 

DESCRIPTION:  Data  identifying  tasks  and/or  subtasks  on  which  specified  airmen 

are  scheduled  to  be  evaluated  for  task-requisite  knowledge  and/or 
task  performance.  These  requirements  may  pertain  to  position 
qualification  or  quality  control  evaluation  events. 

DATA  SOURCE:  Process  1.0,  Manage  Training 

DATA  DESTINATION:  Process  2.0,  Evaluate  Trainees  and  Training  System 

STRUCTURE:  Structure  will  be  determined  in  the  B-l,  B-5,  and  C-5  specifications. 
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DATA  FLOW: 
DESCRIPTION: 

DATA  SOURCE: 
DATA  DESTINATION: 
STRUCTURE: 


DATA  FLOW: 
DESCRIPTION: 

DATA  SOURCE: 

DATA  DESTINATION: 

STRUCTURE: 


Evaluation  Results 

Data  detailing  the  outcomes  of  completed  evaluation  events.  The 
data  will  include  pass/f ail  information.  It  may  also  include  trainee 
perfoimance  on  specific  questions  or  performance  steps. 

Process  2.0,  Evaluate  Trainees  and  Training  System 

Process  1.0,  Manage  Training 

Structure  will  be  determined  in  the  B-l,  B-5,  and  C-S  specifications. 


MTL  Validation  Data 


This  data  flow  consists  of  Tentative  Master  Task  List  (TMTL) 
data  that  is  passed  from  Process  1.0,  Manage  Training,  to  the  Air 
Force  Human  Resources  Laboratory  (AFHRL)  for  validation  and 
the  response  of  AFHRL  concerning  the  validity  of  the  TMTL. 
Upon  being  validated  by  AFHRL,  Process  1.0  will  incorporate  the 
TMTL  data  into  the  Master  Task  List  (MTL). 


Process  1.0,  Manage  Training;  External  Entity  6,  Air  Force  Human 
Resources  Laboratory 

External  Entity  6,  Air  Force  Human  Resources  Laboratory;  Process 
1.0,  Manage  Training 


Structure  will  be  determined  in  the  B-  1 ,  B-5,  and  C-5  specifications. 
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DATA  FLOW:  Operation/Maintenance  Requirements 

DESCRIPTION:  Data  identifying  the  personnel,  maintenance,  and  system  support 

resources  required  by  the  subsystem  for  adequate  functioniing 
within  the  AOTS  system. 


DATA  SOURCE:  Process  1.0,  Manage  Training;  Process  2.0,  Evaluate  Trainees  and 

Training  System-  Process  3.0,  Provide  Computer  Support;  Process 
S.0,  Manage  Training  Development  and  Delivery 

DATA  DESTINATION:  Process  4.0,  Manage  Personnel  and  System  Support 

STRUCTURE:  Structure  will  be  determined  in  the  B-l,  B-5,  and  C-S  specifications. 


DATA  FLOW:  Personnel  Data/Changes 


DESCRIPTION:  Elements  extracted  from  the  Personnel  Data  System  (PDS)  that 

are  required  to  establish  Airman  Training  Records  (ATRs),  to 
manage  training,  and  to  report  training  status  either  by  individual 
or  group.  Also  included  are  Unit  Manager  Personnel  Requirements 
(UMPR),  a  listing  that  shows  manpower  authorizations  for 
organizations  by  workcenter  and  includes  AFSCs  authorized, 
authorized  ranks,  and  position  identification  numbers. 

DATA  SOURCE:  External  Entity  7,  Personnel  Data  System 

DATA  DESTINATION:  Process  1.0,  Manage  Training 


STRUCTURE:  Structure  will  be  determined  in  the  B-l,  B-5,  and  C-5  specifications. 
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DATA  FLOW: 
DESCRIPTION: 

DATA  SOURCE: 
DATA  DESTINATION: 
STRUCTURE: 


DATA  FLOW: 
DESCRIPTION: 

DATA  SOURCE: 
DATA  DESTINATION: 
STRUCTURE: 


Personnel  Qualifications 

Data  provided  to  the  Training  Management  Subsystem  by  the 
Operational  Units  concerning  airman  training  needs,  airman 
progress  toward  position  qualification,  airman  individual  training 
requirements,  and  other  personnel  data  required  to  support  airman 
training  and  airman  training  records. 


External  Entity  5,  Operational  Units 
Process  1.0,  Manage  Training 

Structure  will  be  determined  in  the  B-l,  B-S,  and  C-5  specifications. 


Trainee  Progress  Data 


Data  concerning  individual  trainee  progress  through  training 
materials  contained  within  the  Training  Development  and 
Delivery  Subsystem. 

Process  5.0,  Manage  Training  Development  and  Delivery 
Process  1.0,  Manage  Training 

Structure  will  be  determined  in  the  B-l,  B-5,  and  C-5  specifications. 
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DATA  FLOW: 
DESCRIPTION: 

DATA  SOURCE: 

DATA  DESTINATION: 

STRUCTURE: 


DATA  FLOW: 
DESCRIPTION: 

DATA  SOURCE: 
DATA  DESTINATION: 
STRUCTURE: 


Training  Data/Schedules 

Contains  off-line  training  requirements,  schedules,  completions, 
and  progress  reports  for  individual  airmen. 

External  Entity  8,  Off-Line  Training  Activities;  Process  1.0, 
Manage  Training 

Process  1.0,  Manage  Training;  External  Entity  8,  Off-Line 
Training  Activities 

Structure  will  be  determined  in  the  B-l,  B-S,  and  C-5  specifications. 


Training  Policies 

Contains  Ancillary  and  Additional  Duty  training  requirements. 
External  Entity  4,  Air  Force  Military  Personnel  Center  (AFMPC) 
Process  1.0,  Manage  Training 

Structure  will  be  determined  in  the  B-l,  B-5,  and  C-5  specifications. 
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DATA  FLOW: 
DESCRIPTION: 


DATA  SOURCE: 
DATA  DESTINATION: 
STRUCTURE: 


DATA  FLOW: 
DESCRIPTION: 

DATA  SOURCE: 
DATA  DESTINATION: 
STRUCTURE: 


Training  Reports 


Contains  Standard  or  Ad  Hoc  reports.  Standard  reports  are 
generated  automatically  by  the  system  on  a  periodic  basis. 
Authorized  personnel  may  also  obtain  the  current  standard  reports 
upon  demand  at  any  time.  Standard  reports  are  available  at  four 
organizational  levefs--individual,  workcenter,  unit,  and  base.  Ad 
hoc  reports  are  generated  upon  demand  to  provide  information 
for  authorized  personnel  that  is  not  available  in  the  standard 
reports.  The  content  and  structure  of  an  ad  hoc  report  is  defined 
by  the  system  administrator  and  the  requester  at  the  time  that  it 
is  requested. 

Process  2.0,  Evaluate  Trainees  and  Training  System 
External  Entity  S,  Operational  Units 


Structure  will  be  determined  in  the  B-l,  B-5,  and  C-S  specifications. 


Training  Requirements/Schedules 


Training  requirements  and  schedules  for  individual  airmen  that 
is  passed  f  rom  the  Training  Management  Subsystem  to  the  Training 
Development  and  Delivery  Subsystem  in  order  to  enable  the  airman 
to  proceed  with  training. 


Process  1.0,  Manage  Training 


Process  S.0,  Manage  Training  Development  and  Delivery 
Structure  will  be  determined  in  the  B-l,  B-5,  and  C-5  specifications. 


